TypedDict does not allow optional keys?

Question

I wanted to have a schema validation using pydantic, and also TypedDict to define the part of a nested dict schema. However, I realised that the Optional does not work if it is specified within the TypedDict class.

I read that this class will render all the keys within as required, and a way to make ALL of them as optional was to at total=False. However, I only wanted one of the keys to be optional and the rest required. Is there a way to overcome this limitation?

from typing import List, Optional
from pydantic import BaseModel
from typing_extensions import TypedDict

class _trending(TypedDict):
    allStores: Optional[bool] = False
    category: str
    date: str
    average: List[int]

class RequestSchema(BaseModel):
    storeId: str
    trending: _trending

EDIT

I tried this previously as I thought it is similar to a nested list.

from typing import List, Optional, Dict
from pydantic import BaseModel


class _trending(BaseModel):
    allStores: Optional[bool] = False
    category: str
    date: str
    average: List[int]

class RequestSchema(BaseModel):
    storeId: str
    trending: Dict[_trending]

but faced an error msg saying that Dict requires 2 parameters. Apparently Dict works differently and I can't define it as a class like I hope to be?

Juxtapose

Why do you want to use TypedDict and not BaseModel as the base class for _trending? BaseModel is working for your mentioned use. – Bannerman

Hi, I tried but it does not work. the trending value is a dict, so this is the closest I can get to using a TypedDict class. pydantic-docs.helpmanual.io/usage/types/#typeddict. Updated the question the previous attempt w error. – Juxtapose

oh... I get what u mean now, I am overthinking it~ thanks for the tip! it works :) – Juxtapose

Answer 1

Since Python 3.11, as per PEP 655, what you need is NotRequired:

class _trending(TypedDict):
    allStores: NotRequired[bool]
    category: str
    date: str
    average: List[int]

notice that you shouldn't use Optional in TypedDict, and only use (Not)Required

if you want to use TypedDict with Pydantic, you could refer this article

Thickskinned

typing_extensions (where the future typing stuff live) in GitHub does have NotRequired and Required. As of Feb '22 pypi has only 4.0.1, which does not have these changes. But anyway experimental features means they are buggy (cf. Self Issues) and some drafts do get rejected, so may disappear. – Severity

what to import does matter, especially this case. The code should also import TypedDict from typing_extensions and not from typing. – Endometriosis

What about in Python 3.10? – Dabster

Since Python 3.11 with PEP-655's NotRequired are released now, this should be the accepted answer – Pleonasm

Answer 2

I used this question as duplicate target, but noticed that another option is missing here.

If you don't like NotRequired (for example, if you have many required and many optional keys and don't want to repeat NotRequired many times) or don't want to bother with typing_extensions (rare case), you can tweak totality.

The following definitions of Main* are equivalent:

import sys
# You may also pick one without version check, of course
if sys.version_info < (3, 11):
    from typing_extensions import TypedDict, Required, NotRequired
else:
    from typing import TypedDict, Required, NotRequired


class Main1(TypedDict):
    foo: int
    bar: str
    baz: NotRequired[int]
    qux: NotRequired[str]


class Main2(TypedDict, total=False):
    foo: Required[int]
    bar: Required[str]
    baz: int
    qux: str


class _Main3(TypedDict):
    foo: int
    bar: str

class Main3(_Main3, total=False):
    baz: int
    qux: str


class _Main4(TypedDict, total=False):
    baz: int
    qux: str
    
class Main4(_Main4):
    foo: int
    bar: str

Here's PEP explanation of totality:

The totality flag only applies to items defined in the body of the TypedDict definition. Inherited items won’t be affected, and instead use totality of the TypedDict type where they were defined. This makes it possible to have a combination of required and non-required keys in a single TypedDict type.

Here's example of checking with definitions above:

Main1(foo=1, bar='bar', baz=2, qux='qux')
Main1(foo=1, bar='bar', baz=2)
Main1(foo=1, bar='bar')
Main1(foo=1, baz=2, qux='qux')  # E: Missing key "bar" for TypedDict "Main1"  [typeddict-item]
Main1(foo=1, bar='bar', who=None)  # E: Extra key "who" for TypedDict "Main1"  [typeddict-item]

Main2(foo=1, bar='bar', baz=2, qux='qux')
Main2(foo=1, bar='bar', baz=2)
Main2(foo=1, bar='bar')
Main2(foo=1, baz=2, qux='qux')  # E: Missing key "bar" for TypedDict "Main2"  [typeddict-item]
Main2(foo=1, bar='bar', who=None)  # E: Extra key "who" for TypedDict "Main2"  [typeddict-item]

Main3(foo=1, bar='bar', baz=2, qux='qux')
Main3(foo=1, bar='bar', baz=2)
Main3(foo=1, bar='bar')
Main3(foo=1, baz=2, qux='qux')  # E: Missing key "bar" for TypedDict "Main3"  [typeddict-item]
Main3(foo=1, bar='bar', who=None)  # E: Extra key "who" for TypedDict "Main3"  [typeddict-item]

Main4(foo=1, bar='bar', baz=2, qux='qux')
Main4(foo=1, bar='bar', baz=2)
Main4(foo=1, bar='bar')
Main4(foo=1, baz=2, qux='qux')  # E: Missing key "bar" for TypedDict "Main4"  [typeddict-item]
Main4(foo=1, bar='bar', who=None)  # E: Extra key "who" for TypedDict "Main4"  [typeddict-item]

You can fiddle with this in playground

Hankhanke