Why is Python 3.x's super() magic?

In Python 3.x, super() can be called without arguments:

class A(object):
    def x(self):
         print("Hey now")

class B(A):
    def x(self):
        super().x()

>>> B().x()
Hey now

In order to make this work, some compile-time magic is performed, one consequence of which is that the following code (which rebinds super to super_) fails:

super_ = super

class A(object):
    def x(self):
        print("No flipping")

class B(A):
    def x(self):
        super_().x()

>>> B().x()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in x
RuntimeError: super(): __class__ cell not found

Why is super() unable to resolve the superclass at runtime without assistance from the compiler? Are there practical situations in which this behaviour, or the underlying reason for it, could bite an unwary programmer?

... and, as a side question: are there any other examples in Python of functions, methods etc. which can be broken by rebinding them to a different name?

The new magic super() behaviour was added to avoid violating the D.R.Y. (Don't Repeat Yourself) principle, see PEP 3135. Having to explicitly name the class by referencing it as a global is also prone to the same rebinding issues you discovered with super() itself:

class Foo(Bar):
    def baz(self):
        return super(Foo, self).baz() + 42

Spam = Foo
Foo = something_else()

Spam().baz()  # liable to blow up

The same applies to using class decorators where the decorator returns a new object, which rebinds the class name:

@class_decorator_returning_new_class
class Foo(Bar):
    def baz(self):
        # Now `Foo` is a *different class*
        return super(Foo, self).baz() + 42

The magic super() __class__ cell sidesteps these issues nicely by giving you access to the original class object.

The PEP was kicked off by Guido, who initially envisioned super becoming a keyword, and the idea of using a cell to look up the current class was also his. Certainly, the idea to make it a keyword was part of the first draft of the PEP.

However, it was in fact Guido himself who then stepped away from the keyword idea as 'too magical', proposing the current implementation instead. He anticipated that using a different name for super() could be a problem:

My patch uses an intermediate solution: it assumes you need __class__ whenever you use a variable named 'super'. Thus, if you (globally) rename super to supper and use supper but not super, it won't work without arguments (but it will still work if you pass it either __class__ or the actual class object); if you have an unrelated variable named super, things will work but the method will use the slightly slower call path used for cell variables.

So, in the end, it was Guido himself that proclaimed that using a super keyword did not feel right, and that providing a magic __class__ cell was an acceptable compromise.

I agree that the magic, implicit behaviour of the implementation is somewhat surprising, but super() is one of the most mis-applied functions in the language. Just take a look at all the misapplied super(type(self), self) or super(self.__class__, self) invocations found on the Internet; if any of that code was ever called from a derived class you'd end up with an infinite recursion exception. At the very least the simplified super() call, without arguments, avoids that problem.

As for the renamed super_; just reference __class__ in your method as well and it'll work again. The cell is created if you reference either the super or __class__ names in your method:

>>> super_ = super
>>> class A(object):
...     def x(self):
...         print("No flipping")
... 
>>> class B(A):
...     def x(self):
...         __class__  # just referencing it is enough
...         super_().x()
... 
>>> B().x()
No flipping

Recommended topics

Hot tags