Using modern typing features on older versions of Python

Question

So, I was writing an event emitter class using Python.

Code currently looks like this:

from typing import Callable, Generic, ParamSpec

P = ParamSpec('P')

class Event(Generic[P]):
  def __init__(self):
    ...

  def addHandler(self, action : Callable[P, None]):
    ...

  def removeHandler(self, action : Callable[P, None]): 
    ...

  def fire(self, *args : P.args, **kwargs : P.kwargs):
    ...

As you can see, annotations depend on ParamSpec, which was added to typing in python 3.10 only.

And while it works good in Python 3.10 (on my machine), it fails in Python 3.9 and older (on other machines) because ParamSpec is a new feature.

So, how could I avoid importing ParamSpec when running program or use some fallback alternative, while not confusing typing in editor (pyright)?

Euler

Answer 1

I don't know if there was any reason to reinvent the wheel, but typing_extensions module is maintained by python core team, supports python3.7 and later and is used exactly for this purpose. You can just check python version and choose proper import source:

import sys

if sys.version_info < (3, 10):
    from typing_extensions import ParamSpec
else:
    from typing import ParamSpec

Muntin

I've reinvented the wheel just because I've never heard of it before. Thanks for help! – Euler

Answer 2

This could be solving by wrapping from typing import ... into if TYPE_CHECKING:

if TYPE_CHECKING:
    from typing import Callable, Generic, ParamSpec
else:
    # Fake ParamSpec
    class ParamSpec:
        def __init__(self, _):
            self.args = None
            self.kwargs = None
    # Base class to be used instead Generic
    class Empty:
        pass
    # Thing what returns Empty when called like Generic[P]
    class _Generic:
        def __getitem__(self, _):
            return Empty
    # Callable[anything] will return None
    class _Callable:
        def __getitem__(self, _):
            return None
    # Make instances
    Callable = _Callable()
    Generic = _Generic()

... code

if not TYPE_CHECKING:
    # To allow usage of
    # evt : Event[[int, str]]

    # Store existing Event class
    _Event = Event
    class FakeEvent:
        # Event(...) calls original constructor
        def __call__(self, *args, **kwds):
            return _Event(*args, **kwds)
        # Event[...] returns original Event
        def __getitem__(self, _):
            return _Event
    # Replace Event class
    Event = FakeEvent()

This allows code to be run with older versions of Python while using typing from 3.10 in the editor.

Euler

Answer 3

Contrary to the other answer I advise you to not use a sys.version_info check to decide between typing and typing_extensions, only do it when you are really sure that you need it. If unsure its nearly, but not always safer, to rely on from typing_extensions import <typing_feature> for features that are currently still under frequent development. Its also noteworthy that typing_extensions reexports the standard typing variant if no fix or backport is necessary.

I advise you to consult the typing_extensions documentation or the source code when you use newer features to decide which of the two modules you should use.

In general, typing and typing_extensions are treated the same by type-checkers. However, typing_extensions fixes bugs or backports newer features and syntax to older python versions. Type-checkers implement these changes quite quickly and might not see the error in the below code that will happen during runtime in Python 3.10-3.12.

from typing import ParamSpec
# pyright report the error; mypy does not
DefaultP = ParamSpec("DefaultP", default=[str, int])

In this example typing_extensions.ParamSpec would support the PEP 696 default argument which is planned to come with Python 3.13. The normal typing library does not. from typing_extensions import ParamSpec without version pinning should therefore be preferred.

Another example is Literal which was introduced in Python 3.8 and is only reexported by typing_extensions from Python 3.10 onwards.

---

Some rule-of thumb to help you choose:

• typing_extensions if you need the backport to older versions
• typing_extensions if it fixes bugs in the typing module inside your version range
• typing if you do not need a backport of newer features and its unlikely that others using your code need a backport.
• typing if typing_extensions reexports its for all versions.
• typing | typing_extensions : if typing_extensions is limited in its backport variant (see documentation or issues) and your code would result in runtime failures; in this case if sys.version_info checks can be useful.
• (consider) from other modules like collections.abc for deprecated typing classes.

---

Last but not least in a requirements file do not put a strong pin on typing_extensions: The typing_extensions documentation recommends to use:

typing_extensions >=x.y <(x+1), where x.y is the first version that includes all features you need. [For the near 2024+ future x=4] and we do not expect to increase the major version number [x to 5] in the foreseeable future.

Peneus