Using mypy with with lazy initialization of instance attributes

Question

I'm trying to use mypy in my projects, but many of the instance attributes I use are only initialized after __init__, and not inside it. However, I do want to keep the good practice of declaring all instance attributes at __init__, so I need some complicated solutions to make this work.

An example to how I want this to behave (currently mypy is complaining):

from typing import Optional

class Foo:
    def __init__(self, x: int):
        self.x = x
        self.y: int = None  # will initialize later, but I know it will be an int

    def fill_values(self):
        self.y = self.x**2

    def do(self) -> int:
        return self.x + self.y

Currently mypy complains about the assignment of self.y, and wants it to be Optional or None.

If I agree with it and change the line to self.y: Optional[int] = None, then mypy complains on the return value of do, because self.y might be None.

The only way I found around it is to add as assert before using self.y, like: assert self.y is not None, which mypy picks up and understands. However, starting each method with many asserts is quite hard. I have many such values, and usually one method that initializes all of them, and all other methods runs after it.

I understand that mypy is rightfully complaining (the method do can be called before fill_values), but even when I try to prevent it I can't get mypy to accept this. I can extend this example by adding more functionality but mypy can't infer this:

from typing import Optional

class Foo:
    def __init__(self, x: int):
        self.x = x
        self.y: int = None  # will initialize later, but I know it will be an int

    def fill_values(self):
        self.y = x**2

    def check_values(self):
        assert self.y is not None

    def do(self) -> int:
        if self.y is None:
            self.fill_values()
        self.check_values()
        return self.x + self.y

Any idea of a more elegant solution that multiple assert statements and Optional types which obscure the code?

Bough

If there's no reasonable default for self.y, what would you do if do was called before fill_values? You probably would want to pass this decision back up to where it was being called, by perhaps raising an exception? – Blueweed

I've updated the question to reflect that option, but mypy is still complaining – Bough

You can ignore the wrong assignment via a trailing # type: ignore[assignment] comment, or you can turn this behaviour off completely by passing --no-implicit-optional, but you'll loose the none-checks in the situations like @Blueweed described. As for validation of optionals: mypy only understands direct assertions, not when you put them in a separate method as the inference becomes too cumbersome. – Varistor

Answer 1

I have found this to work for me:

class Foo:
    def __init__(self, x: int):
        self.x = x
        self.y: int  # Give self.y a type but no value

    def fill_values(self):
        self.y = self.x ** 2

    def do(self) -> int:
        return self.x + self.y

Essentially all you are doing is telling mypy that self.y will be an integer when (and if) it is initialised. Trying to call self.y before it is initialised will raise an error and you can check if it has been initialised using hasattr(self, "y").

Run it on mypy playground.

Aftergrowth

That's brilliant! it also solves the issue of "communicating" that self.y is eventually gonna be part of the class without actually initializing it. – Bough

Any reason this is not working with mypy==0.982 and python 3.9? MyPy still complains – Girovard

@Girovard this is most likely due to a change to mypy. Try an older version of mypy. See if you can figure out the exact version where this behaviour changed and check the release notes for that version. – Aftergrowth

it should be self.y = self.x**2 – Nitin

Answer 2

The "declare but not define" idiom leaves me wanting.

The example in this answer doesn't show the problem I find with it.

Take this example instead:

class Lazy:
    def __init__(self) -> None:
        self._file: LargeSlowDatabaseAccessor

    def get_value(self, key: str) -> DataBaseRecord:
        if not hasattr(self, '_file'):
            self._file = connect_to_slow_database()
        return self._file.get(key)

The author of this article doesn't like it because he thinks it's funny that a class doesn't know its own code. I don't like it for a more practical reason: I now have an attribute name trapped in a string. Immediately after writing this code, I realized it should have been called self._data; no refactoring tool I know of will search for hasattr, etc. calls for you. So I may miss that one spot and end up with a class that's always performing the expensive operation that I'm trying to avoid.

Inspired by this comment, one could make use of properties here:

from typing import Optional


class Foo:
    def __init__(self, x: int):
        self.x = x
        self._y: Optional[int] = None  # will initialize later, but I know it will be an int

    @property
    def y(self) -> int:
        if self._y is None:
            self._y = self.x**2
        return self._y

    def do(self) -> int:
        return self.x + self.y

This also works well for other typical lazy initialization things like files or the make-believe LargeSlowDatabaseAccessor.

In other cases there's a newer thing called TypeGuard. See here.

Rebel