How should I use the Optional type hint?

Question

I'm trying to understand how to use the Optional type hint. From PEP-484, I know I can use Optional for def test(a: int = None) either as def test(a: Union[int, None]) or def test(a: Optional[int]).

But how about following examples?

def test(a : dict = None):
    #print(a) ==> {'a': 1234}
    #or
    #print(a) ==> None

def test(a : list = None):
    #print(a) ==> [1,2,3,4, 'a', 'b']
    #or
    #print(a) ==> None

If Optional[type] seems to mean the same thing as Union[type, None], why should I use Optional[] at all?

Zins

What is the advantage of using either Optional or Union[..., None] rather than a: list = None ? Isn't that syntaxe already self explanatory ? – Borrow

@Borrow - You are correct. For keyword args, both mypy and IDEs are able to assume the obvious and automatically treat them as Optional. See my answer below. – Astronomer

@Troy, though this is not recommended. – Exoskeleton

docs.python.org/3/library/typing.html#typing.Optional – Mammilla

in a nutshell: do arg: type | None = None for python>=3.10 – Clockmaker

Answer 1

Optional[...] is a shorthand notation for Union[..., None], telling the type checker that either an object of the specific type is required, or None is required. ... stands for any valid type hint, including complex compound types or a Union[] of more types. Whenever you have a keyword argument with default value None, you should use Optional. (Note: If you are targeting Python 3.10 or newer, PEP 604 introduced a better syntax, see below).

So for your two examples, you have dict and list container types, but the default value for the a keyword argument shows that None is permitted too so use Optional[...]:

from typing import Optional

def test(a: Optional[dict] = None) -> None:
    #print(a) ==> {'a': 1234}
    #or
    #print(a) ==> None

def test(a: Optional[list] = None) -> None:
    #print(a) ==> [1, 2, 3, 4, 'a', 'b']
    #or
    #print(a) ==> None

There is technically no difference between using Optional[] on a Union[], or just adding None to the Union[]. So Optional[Union[str, int]] and Union[str, int, None] are exactly the same thing.

Personally, I'd stick with always using Optional[] when setting the type for a keyword argument that uses = None to set a default value, this documents the reason why None is allowed better. Moreover, it makes it easier to move the Union[...] part into a separate type alias, or to later remove the Optional[...] part if an argument becomes mandatory.

For example, say you have

from typing import Optional, Union

def api_function(optional_argument: Optional[Union[str, int]] = None) -> None:
    """Frob the fooznar.

    If optional_argument is given, it must be an id of the fooznar subwidget
    to filter on. The id should be a string, or for backwards compatibility,
    an integer is also accepted.

    """

then documentation is improved by pulling out the Union[str, int] into a type alias:

from typing import Optional, Union

# subwidget ids used to be integers, now they are strings. Support both.
SubWidgetId = Union[str, int]


def api_function(optional_argument: Optional[SubWidgetId] = None) -> None:
    """Frob the fooznar.

    If optional_argument is given, it must be an id of the fooznar subwidget
    to filter on. The id should be a string, or for backwards compatibility,
    an integer is also accepted.

    """

The refactor to move the Union[] into an alias was made all the much easier because Optional[...] was used instead of Union[str, int, None]. The None value is not a 'subwidget id' after all, it's not part of the value, None is meant to flag the absence of a value.

Side note: Unless your code only has to support Python 3.9 or newer, you want to avoid using the standard library container types in type hinting, as you can't say anything about what types they must contain. So instead of dict and list, use typing.Dict and typing.List, respectively. And when only reading from a container type, you may just as well accept any immutable abstract container type; lists and tuples are Sequence objects, while dict is a Mapping type:

from typing import Mapping, Optional, Sequence, Union

def test(a: Optional[Mapping[str, int]] = None) -> None:
    """accepts an optional map with string keys and integer values"""
    # print(a) ==> {'a': 1234}
    # or
    # print(a) ==> None

def test(a: Optional[Sequence[Union[int, str]]] = None) -> None:
    """accepts an optional sequence of integers and strings
    # print(a) ==> [1, 2, 3, 4, 'a', 'b']
    # or
    # print(a) ==> None

In Python 3.9 and up, the standard container types have all been updated to support using them in type hints, see PEP 585. But, while you now can use dict[str, int] or list[Union[int, str]], you still may want to use the more expressive Mapping and Sequence annotations to indicate that a function won't be mutating the contents (they are treated as 'read only'), and that the functions would work with any object that works as a mapping or sequence, respectively.

Python 3.10 introduces the | union operator into type hinting, see PEP 604. Instead of Union[str, int] you can write str | int. In line with other type-hinted languages, the preferred (and more concise) way to denote an optional argument in Python 3.10 and up, is now Type | None, e.g. str | None or list | None.

Gosser

@MartijnPieters Don't we need to import Dict and List from typing and write Optional[Dict] and Optional[List] instead of Optional[dict]... – Phemia

@Alireza yes, and I state that in my answer already. Look for: Side note: You want to avoid using the standard library container types in type hinting however, as you can't say anything about what types they must contain – Gosser

The importance of the side note about list and dict cannot be stressed enough. I hadn't realized this was a fact after consulting the official Python Documentation, until this discussion. Cheers to you guys for stressing this point. – Fictive

This is the first time I've ever heard of such a thing. Can someone point me towards documentation explaining further why such an avoidance is a good thing? – Croton

Funnily enough, when the new type union operator (PEP 604) will be introduced in Python 3.10, the preferred way will get back to Union[str, int, None], now written in a new fancy way as str | int | None, without the need to import the typing module. – Beaubeauchamp

@Jeyekomon: yes, but take into account that if you write any code that has to work on Python 3.9 or older you still may not be able to use that syntax, depending on where it is used. It'll be a while still before it'll be commonplace. – Gosser

You can use dict[X] and list[X] on Python 3.7+ if you add from __future__ import annotations. – Atahualpa

Does Optional imply that a kwarg argument can be left out of the passed kwargs when calling a function? In function test above: test() (similiar to an interface property in TS having someProperty?: string? – Streamline

@coler-j: yes, Optional explicitly marks the argument as optional, so it can be left out of the call. – Gosser

@MartijnPieters That's not true. def f(a: int, b: Optional[int]) doesn't make the call f(2) valid. – Curch

@quant_dev: that's not the fault of Optional, but of the developer marking an argument as 'optional' when it is not! Type hints do not by themselves enforce their meaning. – Gosser

@MartijnPieters that's not the point. The point is that Optional doesn't mean what you say it means. – Curch

@MartijnPieters Indeed quant_dev is right, Optional does explicitly not mean that the argument is optional, see PEP-484 or docs.python.org/3/library/typing.html#typing.Optional. You can complain about this being a misnomer, but that's how it is. – Madaih

@quant_dev: ick, Marius is correct. Optional only means that the argument can default to None, not that it can be omitted. My mistake. – Gosser

I do not think that leaving a space before and after the = and before the None is needed. See the other answer, and without the typing module, you would also just write it without spaces in between. Thus: Mapping[str, int]]=None, would it be good? – Unrounded

@questionto42: no, the spaces are not required, but the Python code style guide recommends you use spaces there. – Gosser

Answer 2

Directly from mypy typing module docs.

Optional[str] is just a shorthand or alias for Union[str, None]. It exists mostly as a convenience to help function signatures look a little cleaner.

Update for Python 3.10+

you can now use the pipe operator as well.

# Python < 3.10
def get_cars(size: Optional[str]=None):
    pass
# Python 3.10+
def get_cars(size: str|None=None):
    pass

Milissamilissent

It should have been called Nullable. Optional is quite misleading. – Curch

Agreed. It's actually nothing to do with default arguments, which many people think of as 'optional'. I think this is the root of a lot of the confusion about it. – Cram

It's probably called that due to the influence of functional programming and this has affected other languages. Haskell's Option type, Java's Optional, Scala's Option are all good examples. – Protomartyr

@GauthamC. Also C++ std::optional. It looks like a standard name in many programming languages, so I think the name is reasonable. Also, something like Nullable would have been more evocative for Python programmers but unacceptable for computer scientists. – Hygienist

@Curch It's definitely NOT misleading to call it Optional, quite the contrary. – Heidi

@Heidi it's even better if you could at least provide a proper counter-argument. – Fic

@Fic I don't know what a "proper" counter-argument be, but Gautham C. already provided plenty of examples why the notion of an option is pretty much ubiquitous. If it's not enough to you, then I guess I can't help you, sorry. – Heidi

Answer 3

While the accepted answer is the correct answer, one additional thing to note is that, in the context of kwargs, both Optional[...] and Union[..., None] are redundant and unnecessary. If you're immediately setting your kwarg to None, then both mypy and IDEs assume the obvious and automatically treat the arg as Optional[...].

IDE:

mypy:

For variables and method/function return values, Optional[...] is still necessary, however, as mypy cannot know, in those cases, to automatically assume anything.

Astronomer

True, but not recommended. – Exoskeleton

That's only true with the option --no-strict-optional, though, which was the default before version 0.600. – Madaih

Answer 4

Note that since Python 3.10 you can simplify your code and type it like:

def foo(
   bar: int | None = None,
   another_bar: Callable[[int, list, float, datetime | None], str],
):

Quarrel

Answer 5

As already noted in several of the comments, ever since Python 3.7 it has been possible to use the new-style type annotations via a __future__ import:

from __future__ import annotations

def test(a: dict[str, int] | None = None) -> None:
   ...

As for automatically answering this and many other general best-practices questions, I would highly recommend using pyupgrade to automatically reformat your type annotations and the rest of your code with modern Python style. For a single file, after adding the __future__ import, run pyupgrade --py37-plus --keep-runtime-typing file.py.

If you use Git, then pyupgrade can be set up as a pre-commit hook so that your code always stays modern. Here is the block I use:

# Upgrade code style to the specified minimum supported Python version.
- repo: https://github.com/asottile/pyupgrade
  rev: v3.3.1  # Adjust this to the latest version of pyupgrade
  hooks:
  - id: pyupgrade
    # Adjust the following to the minimum supported Python version for your project.
    args:
    - --py37-plus
    - --keep-runtime-typing

Note: The --keep-runtime-typing argument is required in case you use Pydantic, FastAPI, or other similar tools which rely on runtime typing. Otherwise this argument can be safely omitted.

Nightmare

Notice that def test(a: dict[str, int] | None) -> None: results in TypeError: test() missing 1 required positional argument: 'a' when calling test() but def test(a: dict[str, int] | None = None) -> None: runs withot errors when calling test(). – Haggi