Can a simple difference in Python3 variable names alter the way code runs? [duplicate]

Question

This code...

class Person:
    num_of_people = 0

    def __init__(self, name):
        self.name = name
        Person.num_of_people += 1

    def __del__(self):
        Person.num_of_people -= 1

    def __str__(self):
        return 'Hello, my name is ' + self.name

cb = Person('Corey')
kb = Person('Katie')
v = Person('Val')

Produces the following error...

Exception AttributeError: "'NoneType' object has no attribute 'num_of_people'" in <bound method Person.__del__ of <__main__.Person object at 0x7f5593632590>> ignored

But this code does not.

class Person:
    num_of_people = 0

    def __init__(self, name):
        self.name = name
        Person.num_of_people += 1

    def __del__(self):
        Person.num_of_people -= 1

    def __str__(self):
        return 'Hello, my name is ' + self.name

cb = Person('Corey')
kb = Person('Katie')
vb = Person('Val')

The only difference I see is the last variable name is "vb" vs. "v".

I am leaning Python and am working on the OOP stuff now.

Schuler

@StevenRumbalski: In short, yes. But only at interpreter exit. – Caskey

The first code does not produce that exception. Show your full traceback. (Correction: it doesn't produce that exception in Python 3.3 or higher. In 3.2 it does.) – Tva

@Tva Nah! That's what i was missing.. – Treponema

@Wooble: That's because dictionary hashing for strings is randomized in 3.3. It'll happen in 3.3 too, when the stars align just right for the keys to collide in the right order. In other words, re-run your test several times and you'll see it happen on some runs. – Caskey

@Wooble: Last but not least, you certainly won't see this error in CPython 3.4, as it has a new safe object finalization codepath that removes the reason for this error altogether. – Caskey

Note: I wouldn't say that what you see is an "error". It's a ignored exception, and it is also a correctly ignored exception, and as such I'd say it's a warning more then an error. If you didn't mess with __del__ you wouldn't have this problem, and I believe the code you have written is misusing __del__. In fact in python<3.4 you could easily create some cycles of references so that your num_of_people counter is off. – Pushover

Not just Python3. In Python 2.7.3 I get two such errors for the first example and one for the second. – Terbium

Answer 1

Yes, although it is not the so much the variable name that causes this, not directly.

When Python exits, all modules are deleted too. The way modules are cleaned up is by setting all globals in a module to None (so those references no longer refer to the original objects). Those globals are keys in a dictionary object, and as dictionaries are ordered arbitrarily, renaming one variable can change the order in which variables are cleared.

When you renamed v to vb, you altered the order in which variables are cleared, and now Person is cleared last.

One work-around is to use type(self).num_of_people -= 1 in the __del__ method instead:

def __del__(self):
    type(self).num_of_people -= 1

because the instance will always have a reference to the class still, or test if Person is not set to None:

def __del__(self):
    if Person is not None:
        Person.num_of_people -= 1

Two notes:

• CPython 3.4 no longer sets globals to None (in most cases), as per Safe Object Finalization; see PEP 442.

• CPython 3.3 automatically applies a randomized hash salt to the str keys used in a globals dictionary; this makes the behaviour you observed even more random, merely re-running your code several times may or may not trigger the error message.

Caskey

Ugh.. The answer is brilliant, but I don't really understand this. What do you mean by 'on intepreter exit'? Can you please explain? – Treponema

@aIKid: When Python exits, (or if you delete the module by removing all references to it and deleting it from sys.modules, but that's not often done), the module __del__ is called, which then proceeds by first clearing all globals in the module by rebinding their names to None. – Caskey

So the exception is only raised when we exit the interpreter.. <- That's false, right? – Treponema

@aIKid: this should only happen when you exit the interpreter, or you delete the module explicitly. – Caskey

Wow. I don't know there's such thing! Well, thanks! – Treponema

This kind of stuff is why I love StackOverflow. +1, very interesting. – Harakiri

great technical answer, could you summarize it at a higher level for me? – Schuler

@Corey: there isn't really a higher level here; the bottom line is that during finalization, globals like Person and v and vb have None assigned to them, so you cannot count on Person being the class still in __del__. – Caskey

@sds: finalization is finicky; by setting globals to None many common circular references are broken properly without invoking (slow and entirely unneeded) resizes of the globals dictionaries. The finally did find a better way for Python 3.4, so this whole thing is no longer a problem with that version. – Caskey