How to use static type checking using Dict with different value types in Python 3.6?

Question

Trying to use static types in Python code, so mypy can help me with some hidden errors. It's quite simple to use with single variables

real_hour: int = lower_hour + hour_iterator

Harder to use it with lists and dictionaries, need to import additional typing library:

from typing import Dict, List
hour_dict: Dict[str, str] = {"test_key": "test_value"}

But the main problem - how to use it with Dicts with different value types, like:

hour_dict = {"test_key": "test_value", "test_keywords": ["test_1","test_2"]}

If I don't use static typing for such dictionaries - mypy shows me errors, like:

len(hour_dict['test_keywords'])
- Argument 1 to "len" has incompatible type

So, my question: how to add static types to such dictionaries? :)

Bulldozer

They're still working on that. – Jinajingle

If all of the types that the dict could contain are known, you could use Union. e.g. Dict[str, Union[list, str]]. However, this doesn't ensure that particular keys are always of a specified type, it just allows values to be either (e.g.) strings or lists. – Muldoon

(Also, it's Dict[str, str], not Dict[str:str].) – Jinajingle

Related: Python 3 dictionary with known keys typing – Domenicadomenico

Answer 1

You need a Union type, of some sort.

from typing import Dict, List, Union

# simple str values
hour_dict: Dict[str, str] = {"test_key": "test_value"}

# more complex values
hour_dict1: Dict[str, Union[str, List[str]]] = {
    "test_key": "test_value", 
    "test_keywords": ["test_1","test_2"]
}

In general, when you need an "either this or that," you need a Union. In this case, your options are str and List[str].

There are several ways to play this out. You might, for example, want to define type names to simplify inline types.

OneOrManyStrings = Union[str, List[str]]

hour_dict2: Dict[str, OneOrManyStrings] = {
    "test_key": "test_value", 
    "test_keywords": ["test_1","test_2"]
}

I might also advise for simplicity, parallelism, and regularity to make all your dict values pure List[str] even if there's only one item. This would allow you to always take the len() of a value, without prior type checking or guard conditions. But those points are nits and tweaks.

Belita

Additional question: how to use static typing with datetime objects? – Bulldozer

@Bulldozer Once a type is defined or imported, it can be used in typed Python just like a builtin such as int or str. So after from datetime import datetime you can just say right_now: datetime = datetime.utcnow(). Or you can use datetime in Unions or as the subtype of List and Dict, or otherwise used just as primitive types can be used. – Belita

Enuice Ok, understand, thanks. Still, mypy doesn't like Union: I define dict as order_data: Dict[str, Union[str, int, List[str]]], but when I try to take data from it (keywords_to_load: List[str] = order_data['keywords']) mypy says "Incompatible types, expression has type Union[str, int, List[str]], variable has type List[str]. Am I doing something wrong? I mean, it's not really a problem, still want to fix as may errors as possible :) – Bulldozer

@Bulldozer The problem is that mypy cannot guarantee the safety of that operation. Yes, it might be a List[str] value retrieved from order_data['keywords']. But it might equally be a str or an int. Mypy can't know, therefore it's not passing the checks. Try something like this instead. That is a type equation mypy can guarantee. – Belita

when you use python 3.11, you can create a union easily by doing e. g. [str|int], so with the | character. – Jobye

Answer 2

While using Union is indeed one way of doing it, a more precise solution would be to use the (currently experimental) TypedDict type, which lets you assign specific types per each string key.

In order to use this type, you must first install the mypy_extensions 3rd party library using pip. You can then do the following:

from typing import List
from mypy_extensions import TypedDict

MyDictType = TypedDict('MyDictType', {
        'test_key': str, 
        'test_keywords': List[str],
})

hour_dict: MyDictType = {
    "test_key": "test_value", 
    "test_keywords": ["test_1","test_2"]
}

Note that we need to explicitly denote hour_dict of being of type MyDictType. A slightly cleaner way of doing this is to use MyDictType as a constructor -- at runtime, MyDictType(...) is exactly equivalent to doing dict(...), which means the below code behaves exactly identically to the above:

hour_dict = MyDictType(
    test_key="test_value", 
    test_keywords=["test_1","test_2"]
)

Finally, note that there are a few limitations to using TypedDict:

1. It's useful only when the dict will contain specific keys with types that are all known at compile time -- you should use regular Dict[...] if you expect a truly dynamic dict.
2. The keys must all be strings.
3. At time being, this type is understood only by mypy (though I understand there are plans to eventually add TypedDict to PEP 484 once it's a little more battle-tested, which would mean any PEP 484 compliant typechecker would be required to support it).

(TypedDict was designed to make it easier to work with JSON blobs/dicts when writing serialization/deserialization logic, which is the reason for these constraints.)

Tragopan