Python dynamic type hinting

Question

I would like to be able to provide a function with a tuple of types which would then be used to deserialize the returned data from a transaction. The function would then return the instances of those types, if the transaction had succeeded. For example:

T = TypeVar('T')

class Base:
    @classmethod
    def deserialize(cls: Type[T], bts: bytes) -> T:
        return cls(**json.loads(bts))

    @classmethod
    def transaction(cls, *types):
        items = self.db.transact([t.generate_load_operation() for t in types])
        items = [t.deserialize(item) for item in items]

        # how do I type-hint transaction, so that it would imply that
        # it will always return a tuple (or a list) of instances of classes 
        # contained in variable types?
        return items

class A(Base):
    pass

class B(Base):
    pass

a_inst, b_inst = Base.transaction(A, B)

How should I go about annotating transaction so that the type-checker could properly infer the types of values returned from it?

Jeraldjeraldine

Type hints exist to support static analysis. They have no effect at runtime and the intention is that they never will. So if there were to be dynamic hinting, there would be nobody or nothing to read it and act on it at runtime. – Bratwurst

@BoarGules, can you, please - provide any links to the information saying "they never will"? Architecturally nothing stops one from providing a function, that given the functions arguments, would return it's return type - this is just another version of the contract systems implemented in other languages. So please do. – Jeraldjeraldine

PEP 484: Non-goals While the proposed typing module will contain some building blocks for runtime type checking -- in particular the get_type_hints() function -- third party packages would have to be developed to implement specific runtime type checking functionality, for example using decorators or metaclasses. Using type hints for performance optimizations is left as an exercise for the reader. – Bratwurst

@Bratwurst I don't believe you understand what I am speaking about. I am looking for static inference based on the constant values supplied to the function. Please note I am giving this in the example - a sufficiently advanced static checker must be able to understand that the value supplied to the function is a constant, therefore it can be checked without actually running the code. – Jeraldjeraldine

@AndreyCizov did you manage to get an answer, I am also looking for something similar . – Tl

There is no way to do that right now @Tl – Jeraldjeraldine

Since you have now rejected two answers that would appear to satisfy the question, citing requirements not stated in the question, please update the question to clarify what you are looking for. If you want "a plug-in for mypy rather than the viability of the said functionality in default python setup" the question should explicitly say as such. – Dennard

If you want your question to be approved you need to explicitly state with references why this is not viable in the current state of the art mypy and then suggest a workaround that you have just mentioned. Again, you have not provided a general solution but a workaround. – Jeraldjeraldine

Answer 1

There is no general way to do this: Variadic arguments such as *types do not preserve order as far as static type checking is concerned. Variadic generics cover this behaviour but are only proposals so far.

One can either use a single annotation for variadic arguments, which degrades to the common base type, or use several annotations requiring to enumerate the most common cases.

---

Annotate the variadic arguments as a type variable (possibly bound via Base). This will infer all types to the same, most common base type.

class Base:
    @classmethod
    def transaction(cls, *types: Type[T]) -> List[T]: ...

class A(Base): ...

class B(Base): ...

reveal_type(Base.transaction(A, A))  # builtins.list[mt.A*]
reveal_type(Base.transaction(A, B))  # builtins.list[mt.Base*]

This is sufficient when using similar types (e.g. just A's or subclasses of A's) and expecting only general features (e.g. just A's). This will not be sufficient when using mixed type (e.g. A and B) since it will degenerate to the common base type (e.g. Base).

---

Provide multiple @overload signatures for a reasonable number of arguments, and use a variadic catch-all for huge number of arguments. This will infer the proper type for the specified cases, and use the most common base type otherwise.

class Base:
    # explicitly enumerated types
    @overload
    @classmethod
    def transaction(cls, t1: Type[T1], /) -> Tuple[T1]: ...
    @overload
    @classmethod
    def transaction(cls, t1: Type[T1], t2: Type[T2], /) -> Tuple[T1, T2]: ...
    # catch all number of types
    @overload
    @classmethod
    def transaction(cls, *ts: Type[T]) -> Tuple[T, ...]: ...

    # implementation
    @classmethod
    def transaction(cls, *types: Type[T]) -> Tuple[T, ...]: ...

class A(Base): ...

class B(Base): ...

reveal_type(Base.transaction(A))        # Revealed type is 'Tuple[mt.A*]'
reveal_type(Base.transaction(A, A))     # Revealed type is 'Tuple[mt.A*, mt.A*]'
reveal_type(Base.transaction(A, B))     # Revealed type is 'Tuple[mt.A*, mt.B*]'
reveal_type(Base.transaction(A, B, A))  # Revealed type is 'builtins.tuple[mt.Base*]'

The limit here is only how many cases one considers to be relevant. This mechanism is also used for the standard library.

Dennard

The question is about dynamic type hinting and whether it is possible to add this as let’s say, a plug-in for mypy rather than the viability of the said functionality in default python setup – Jeraldjeraldine

@AndreyCizov Then the question should state as such. Currently, it says "How should I go about annotating transaction so that the type-checker could properly infer the types of values returned from it?" which is exactly what this answer provides. – Dennard

You have not provided a general solution – Jeraldjeraldine

@AndreyCizov Well, that would be a weird thing to do after asserting no such solution exists. Should I just leave it at that assertion and remove the solutions which work for arbitrary many but not infinitely many arguments? – Dennard

@AndreyCizov Added a reference for an open mypy proposal stuck since years. Conveniently includes pretty much exactly your usecase of transforming a sequence of types to a sequence of instances. – Dennard