Type Hints Convention for Instance Variables Python

Question

I'm unsure of the Python convention for type hinting instance variables - I've been doing them within the __init__ constructor arguments like seen here:

class LoggedVar(Generic[T]):
    def __init__(self, value: T, name: str, logger: Logger) -> None:
        self.name = name
        self.logger = logger
        self.value = value`

But I also see the PEP conventions of annotating instance variables as such (snippet below) and then also doing type hinting within the __init__ arguments:

class BasicStarship:
    captain: str = 'Picard'               # instance variable with default
    damage: int                           # instance variable without default
    stats: ClassVar[Dict[str, int]] = {}  # class variable`

    def __init__(self, damage: int, captain: str = None):
        self.damage = damage
        if captain:
            self.captain = captain  # Else keep the default

Lastly, later on in the PEP 526 article they say one can do the following for convenience and convention:

class Box(Generic[T]):
    def __init__(self, content):
        self.content: T = content

(Both of the above code snippets are from here.)

So — is one of these conventions better/more widely accepted than the others that I should try to stick to (better readability etc..)?

Siamese

I was unaware of PEP 526 until now; thank you. – Annabal

why are captain and damage instance variables in the second example? Aren't they class variables as well? Or is the fact, that they are altered in the init method making them instance variables? If I would have a list, and would alter it with list.append() that alteration would be shared over all instances, so it would still be a class variable. – Poland

@Poland it is all up to whether they're addressed with self or BasicStartship. More accurately, whether they're addressed by an instance or class. For example, outside the class, if you say BasicStarship().captain = 'Blackbeard', you are not changing the default value of 'Picard'. (At least in Python 3.12; I was definitely confused by this from my Python 2.x days.) – Cullan

Answer 1

I would recommend using the first version, where you assign types to your __init__ method's parameters, for most circumstances.

That particular method has the least amount of redundancy while still allowing type checkers to verify that you're calling that __init__ method correctly elsewhere in your code.

I would recommend using either the second or third version, where you explicitly annotate your fields (inside or outside __init__) when your __init__ method has grown complex enough to the point where one or more of the following apply:

1. It's no longer so straightforward what exactly your fields are to begin with
2. There's no longer a one-to-one mapping between your parameters and your fields
3. You have complex initialization logic that obscures how your fields are being assigned.

However, it was unclear to me whether the second or third version was preferred -- I personally prefer the third version because it's more conceptually cleaner and doesn't seem to mix the notion of instance vs class attributes, but I can't deny the second version looks cleaner.

I asked about it on the 'typing' gitter channel, and got the following response from Guido (who, on the off-chance you didn't know, made Python and is currently working on mypy and typing related stuff):

There seem to be strong opinions either way. I do indeed prefer putting attribute annotations in the class body rather than sprinkling them throughout __init__ and other methods. I also think that with PEP 526 this will be the future (also with things like class-based NamedTuple declarations and possibly https://github.com/ericvsmith/dataclasses).

(link to quote)

So, it seems like the second version is recommended over the third, and that defining classes in that manner will become more deeply integrated into the Python language itself at some point in the future!

Edit: PEP 557, data classes was recently accepted and appears to be on-track (?) to be included with Python 3.7.

Dryasdust

Answer 2

@Asara

It seems that as of Python 3.8.10 / Mypy 0.910 (Sep 2021), when it comes to distinguishing between a type annotation for an instance variable in a class definition and a declaration for a class (static) variable in a class definition, the assignment of a default value makes all the difference. If you do not assign a default (e.g., x: int), Python treats the expression as a type annotation; if you assign a default (e.g., x: int = 42), Python treats the expression as a class (static) variable declaration.

It is possible to create a type annotation for a class (static) variable in a class definition, using the ClassVar syntax. If you do not assign a default (e.g., y: ClassVar[int]), a factual class (static) variable will not be created; if you assign a default (e.g., y: ClassVar[int] = 69), a factual class (static) variable will be created.

Brace

If the assignment of a default value makes the difference, why in the above example (taken from PEP 526) the captain variable that does have a default value 'Picard' assigned is described as instance variable with default and not considered a class variable? – Inclining

@pawel: Yeah I think that's confusing, too. captain is definitely a class variable (I tested it). My best guess is that they refer to it as instance variable with default because that's how it's being used. Notice that the constructor doesn't set self.captain if no value was provided. But self.captain and some_instance_of_BasicStarship.captain still work (and return "Picard") because it looks through to the value on the class. – Mccown

I think it's indeed confusing. This simple example would be helpful though - @MarkDoliner – Electrokinetic

If you do not assign a default (e.g., x: int) mypy will still treat it as static – Reiners

Answer 3

I'd stick with what you're doing in LoggedVar, it follows the same rules as everywhere else in Python, so there's less confusion all round.

The BasicStarShip class changes the scope of the variables by moving them out of the __init__ function. With captain declared there, BasicStarShip.captain, will return 'Picard'.

The PEP 526 annotations are nice to read, but it's a new rule for one specific case, the __init__ function. From the Zen of Python:

"Special cases aren't special enough to break the rules."

Hostage

Answer 4

@Gary and for whoever gets confused,

I think the difference comes from using self or ClassName.

Try running these two examples: (both use default value cnt = 0, so it should be class variable)

live code

1. self.cnt += 1 -> works like an instance variable, although a value was assigned(cnt = 0).

class Test:
    cnt = 0 
    def __init__(self):
        # Test.cnt+=1 
        self.cnt+=1 
    def p(self):
        print(f'self.cnt:{self.cnt}, Test.cnt:{Test.cnt}') 

t1 = Test() 
t2 = Test() 
t1.p() 
t2.p() 

# outputs:
# self.cnt:1, Test.cnt:0
# self.cnt:1, Test.cnt:0

---

2. Test.cnt += 1 -> works as a class variable, as expected. (since a value was assigned)

class Test:
    cnt = 0 
    def __init__(self):
        Test.cnt+=1 
        # self.cnt+=1 
    def p(self):
        print(f'self.cnt:{self.cnt}, Test.cnt:{Test.cnt}') 

t1 = Test() 
t2 = Test() 
t1.p() 
t2.p() 

# outputs:
# self.cnt:2, Test.cnt:2
# self.cnt:2, Test.cnt:2

(No type annotation for simplicity - Results are the same with cnt: int = 0)

Induct

Answer 5

I personally prefer to put class variables at class level and instance variables inside the __init__. I would consider doing anything else a "hack", or something that leads to confusion, and here's why:

Just adding a typehint in Python does nothing:

class Empty:
    x: int

assert hasattr(Empty, 'x') is False

It doesn't matter where (module-, class-, or method- level) you do this; the line is ignored.

ClassVar exists, but not InstanceVar

You can declare something a class variable using ClassVar (only for the sake of type checking). Then e.g. Mypy will give you an error when you're trying to modify a class variable via instance (e.g. self.class_var = new_val).

However, Mypy won't give you an error if you modify what you intended to be an instance variable at class level. That's because it can't really tell what's an instance variable and what's a "class-level default", which is what captain is in the example given.

And really, the only reason that self.captain = captain does not change the value of BasicStarship.captain is because of the way string assingment works (it makes copies).

a_str = 'something'
another_str = a_str
another_str += ' else'

assert a_str == 'something'
assert another_str == 'something else'

A list is an example of something that does not work that way:

a_list = ['something']
another_list = a_list
another_list += [' else']
assert a_list == ['something', ' else']
assert another_list == ['something', ' else']

Lists do not get copied on assignment, so you're modifying a single object here.

This leads to confusion if you've convinced yourself that "class variables are anything addressable via the class object and instance variables are anything addressable via instance object". Let's take the approach of "I want to declare all class- and instance-variable defaults at class-level":

class Something:
    instance_str: str = 'instance'

    instance_list: list[str] = []

    class_str: ClassVar[str] = 'class'

    def __init__(self, instance_suffix: str | None = None) -> None:
        if instance_suffix is not None:
            # I think of this as the equivalent of:
            # self.instance_str = Something.instance_str + instance_suffix
            self.instance_str += instance_suffix
            # This has unintended consequences! It's the equivalent of
            # self.instance_list = Something.instance_list # a reference
            # self.instance_list.append(instance_suffix) # modify the original!
            self.instance_list += [instance_suffix]

    def reveal(self) -> None:
        print(f'      {self.instance_str=}')
        print(f'     {self.instance_list=}')
        print(f' {Something.instance_str=}')
        print(f'{Something.instance_list=}')
        print(f'         {self.class_str=}')
        print(f'    {Something.class_str=}')

If we could have annotated instance_list as an InstanceVar, then Mypy could have told us we're inadvertently modifying the Something.instance_list in the constructor. But it can't. There's no magic here; it all has to do with how the assignment operator works for a particular type; strings do what you want but lists do not. You can confirm this with the following code:

print('\n\n')
s1 = Something('_s1')
s2 = Something('_s2')
print('\ns1:')
s1.reveal()  # Shows changes to `Something.instance_list`; bad!
print('\ns2:')
s2.reveal()  # Shows changes to `Something.instance_list`; bad!

print('\n\n')
print('s1.class_str = \'foo\'')
s1.class_str = 'foo'  # mypy error: Cannot assign to class variable "class_str" via instance
print('\ns1:')
s1.reveal()  # shows change to `self.class_str` but not `Something.class_str`
print('\ns2:')
s2.reveal()  # shows no changes

Dissecting BasicStarship

The OP posted this example:

class BasicStarship:
    captain: str = 'Picard'               # instance variable with default
    damage: int                           # instance variable without default
    stats: ClassVar[Dict[str, int]] = {}  # class variable`

    def __init__(self, damage: int, captain: str = None):
        self.damage = damage
        if captain:
            self.captain = captain  # Else keep the default

Here's one way to think about what is happening. At class level:

• BasicStarship.captain exists and its value is 'Picard'. Thus it is a class variable.
• BasicStarship.damage does not exist
• BasicStarship.stats exists and is an empty dictionary; and again, it's a class variable. In the constructor:
• self.damage is created. This does not by any means create a BasicStarship.damage.
• self.captain is created if a captain value is passed in. It does not matter that BasicStarship.captain exists; this is entirely the same as self.damage = damage. If captain is not passed in, self.captain is not created, but hasattr(BasicStarship(), 'captain') returns True because it will "reach back" into the class looking for that (it's not in BasicStarship().__dict__).
• BasicStarship.stats exists and care must be taken in how you handle it. If you suddenly say self.stats = {'kills': 0}, you're creating a new attribute; Mypy will give you an error (because of ClassVar) but you're safe if you ignore it. If you say self.stats['kills'] = 0, you're modifying BasicStarship.stats; Mypy will give you an error and if you ignore it, you have a bug.

This last point is "obvious" in retrospect but I think it requires way too much attention to detail while scanning code. It's also the reason many discourage the use of mutables as function parameter defaults.

For that reason, if someone insisted on advertising instance level attributes at the top of a class, I'd write BasicStarship like this:

class BasicStarship:
    # Instance variables
    ####################
    captain: str
    damage: int

    # Class variables
    #################
    stats: ClassVar[dict[str, int]] = {}

    def __init__(self, damage: int, captain: str = 'Picard'):
        self.damage = damage
        self.captain = captain

To be honest, though, I would only annotate instance variables at class level if a) advertising them leads to clarity, i.e. the constructor is huge and complicated, and b) it avoid creating a trivial constructor when subclassing. Otherwise I would just leave that stuff in the constructor. If you use ClassVar, it's obvious what's a class variable, so this is cleaner to me:

class BasicStarship:
    stats: ClassVar[dict[str, int]] = {}

    def __init__(self, damage: int, captain: str = 'Picard'):
        self.damage = damage
        self.captain = captain

Or perhaps, more succinctly:

The constructor gives you access to the instance via the first parameter, so how could you possibly define instance variables at class level, where you, by definition, do not have access to an instance?

Cullan

Annotating variables on self inside __init__ is not a universal solution, and doing so leads to difficult to read code for all but the simplest of classes. (1) What goes on inside __init__ is a runtime implementation detail, and you can’t do such annotations in .pyi stubs because function / method definitions in there have no body. (2) You can’t do such annotations if your instance is initialised by __new__, metaclass __call__, or some other initialisation method instead of __init__, such as extra steps in @classmethod constructors. – Inpour