Are Mixin class __init__ functions not automatically called?

Question

I'd like to use a Mixin to always add some init functionality to my child classes which each inherit from different API base classes. Specifically, I'd like to make multiple different child classes that inherit from one of these different API-supplied base classes and the one Mixin, which will always have the Mixin initialization code executed in the same way, without code replication. However, it seems that the __init__ function of the Mixin class never gets called unless I explicitly call it in the Child class's __init__ function, which is less than ideal. I've built up a simple test case:

class APIBaseClassOne(object):
    def __init__(self, *args, **kwargs):
        print (" base ")

class SomeMixin(object):
    def __init__(self, *args, **kwargs):
        print (" mixin before ")
        super(SomeMixin, self).__init__(*args, **kwargs)
        print (" mixin after ")

class MyClass(APIBaseClassOne):
    pass

class MixedClass(MyClass, SomeMixin):
    pass

As you can see in the following output, the Mixin function's init never gets called:

>>> import test
>>> test.MixedClass()
 base
<test.MixedClass object at 0x1004cc850>

Is there a way to do this (have an init function in a Mixin get called) without writing every child class to explicitly invoke the Mixin's init function? (i.e., without having to do something like this in every class:)

class MixedClass(MyClass, SomeMixin):
    def __init__(*args, **kwargs):
        SomeMixin.__init__(self, *args, **kwargs)
        MyClass.__init__(self, *args, **kwargs) 

Btw, if all my child classes were inheriting from same base class, I realize I could create a new middle class that inherits from the base class and the mixin and keep it DRY that way. However, they inherit from different base classes with common functionality. (Django Field classes, to be precise).

Lucilucia

In general, using multiple inheritance with base classes that weren't designed for it is a bad idea. Mix-in classes are usually designed together, and mixing-in arbitrary classes produces such messes. In any case, if both base classes each have an __init__ method, how should the interpreter know which one to call, or in which order to call them? – Hejira

@André Caron: It could determine the order like C++ does, where base classes are initialized in declaration order. – Promethium

Answer 1

Sorry I saw this so late, but

class MixedClass2(SomeMixin, MyClass):
    pass

>>> m = MixedClass2()
 mixin before 
 base 
 mixin after

The pattern @Ignacio is talking about is called cooperative multiple inheritance, and it's great. But if a base class isn't interested in cooperating, make it the second base, and your mixin the first. The mixin's __init__() (and anything else it defines) will be checked before the base class, following Python's MRO.

This should solve the general question, though I'm not sure it handles your specific use. Base classes with custom metaclasses (like Django models) or with strange decorators (like @martineau's answer ;) can do crazy things.

Yardage

Even if it was late, this boils it down nicely (although the other answers provide some really good teaching)-- ultimately, an understanding of Python MRO seems critical before embarking on any multiple-inheritance adventures, or unexpected things will happen. THanks! – Lucilucia

Welcome! Absolutely right, it's unfortunate the the MRO is presented as an advanced topic :/ – Yardage

Answer 2

Have the base class invoke super().__init__() even though it is a subclass of object. That way all the __init__() methods will be run.

class BaseClassOne(object):
    def __init__(self, *args, **kwargs):
        super(BaseClassOne, self).__init__(*args, **kwargs)
        print (" base ")

Dy

Thanks! I didn't realize that calling super would cause all __init__() methods to be run. Unfortunately BaseClassOne is an API (Django) supplied base class (i've updated my question to reflect this) but this may get me started in the right direction. – Lucilucia

It won't cause all of them to be run, but it will cause the next one to be run, even if it's in a mixin. – Dy

In a multiple-inheritance situation, all classes should use super() to find the next class to initialize. – Faxan

@kindall: I think many have missed the point that the OP can't change BaseClassOne because it part of an API they have no control over. – Promethium

@martineau: That fact was revealed only after I posted this answer. And that doesn't make this answer incorrect, just much less relevant for the situation. – Dy

Even if you can't change one of the clasess, you could potentially subclass it and give the subclass the desired behavior, or use a wrapper class for the same purpose. Of course, if you have some code you can't change that's a stickler for a particular type, that may not work so well... – Faxan

@kindall: I tried adding an __init__ with a super(...).__init__ call to both the OP's MyClass and MixedClass but the one for SomeMixin is still not called when I instantiate a MixedClass object -- so perhaps I'm misunderstanding what you're suggesting. – Promethium

I think for multiple inheritances, you should call all init of parents that you want. FG: Child(Mixin1, Mixin2):, if you want to call both parent's init you should call them. (super(Mixin1, self).__init__() and super(Mixin2,self).__init__()) otherwise, python will call the first one (Mixin1) – Propriety

I think only this is CORRECT answer here. I will make separate answer to explain it. – Astoria

Answer 3

Python performs no implicit calls to the __init__ methods of a class' super-class(es)—but it's possible to make it happen automatically. One way is by defining a metaclass for your mixed class(es) that creates or extends the mixed class' __init__ method so that it calls all the listed bases' __init__ functions in the order they were listed.

A second way is to do it is to use a class decorator—which is shown in the Edit section below.

Using a metaclass:

class APIBaseClassOne(object):  # API class (can't be changed)
    def __init__(self, *args, **kwargs):
        print('  APIBaseClassOne.__init__()')

class SomeMixin(object):
    def __init__(self, *args, **kwargs):
        print('  SomeMixin.__init__()')

class MixedClassMeta(type):
    def __new__(cls, name, bases, classdict):
        classinit = classdict.get('__init__')  # Possibly None.

        # Define an __init__ function for the new class.
        def __init__(self, *args, **kwargs):
            # Call the __init__ functions of all the bases.
            for base in type(self).__bases__:
                base.__init__(self, *args, **kwargs)
            # Also call any __init__ function that was in the new class.
            if classinit:
                classinit(self, *args, **kwargs)

        # Add the local function to the new class.
        classdict['__init__'] = __init__
        return type.__new__(cls, name, bases, classdict)

class MixedClass(APIBaseClassOne, SomeMixin):
    __metaclass__ = MixedClassMeta  # important
    # If exists, called after the __init__'s of all the direct bases.
    def __init__(self, *args, **kwargs):
        print('  MixedClass.__init__()')

print('MixedClass():')
MixedClass()

Output:

MixedClass():
  APIBaseClassOne.__init__()
  SomeMixin.__init__()
  MixedClass.__init__()

Edit

Here's how to accomplish the same thing with a class decorator (requires Python 2.6+):

class APIBaseClassOne(object):  # API class (can't be changed)
    def __init__(self, *args, **kwargs):
        print('  APIBaseClassOne.__init__()')

class SomeMixin(object):
    def __init__(self, *args, **kwargs):
        print('  SomeMixin.__init__()')

def mixedomatic(cls):
    """ Mixed-in class decorator. """
    classinit = cls.__dict__.get('__init__')  # Possibly None.

    # Define an __init__ function for the class.
    def __init__(self, *args, **kwargs):
        # Call the __init__ functions of all the bases.
        for base in cls.__bases__:
            base.__init__(self, *args, **kwargs)
        # Also call any __init__ function that was in the class.
        if classinit:
            classinit(self, *args, **kwargs)

    # Make the local function the class's __init__.
    setattr(cls, '__init__', __init__)
    return cls

@mixedomatic
class MixedClass(APIBaseClassOne, SomeMixin):
    # If exists, called after the __init__'s of all the direct base classes.
    def __init__(self, *args, **kwargs):
        print('  MixedClass.__init__()')

print('MixedClass():')
MixedClass()

Notes

For Python < 2.6, use MixedClass = mixedomatic(MixedClass) following the class definition.

In Python 3 the syntax for specifying metaclasses is different, so instead of the:

class MixedClass(APIBaseClassOne, SomeMixin):
    __metaclass__ = MixedClassMeta  # important

shown above, you would need to use:

class MixedClass(APIBaseClassOne, SomeMixin, metaclass=MixedClassMeta):

The class decorator version will work as-is in both versions.

Promethium

Thanks for demonstrating this. While the discussion in the other answer has made me rethink whether its a good idea to begin with, this implementation works and I've learned a lot. – Lucilucia

@Ben: Yes, mixins are somewhat controversial, and even considered harmful by some, plus using them in Python is somewhat complicated by the fact that it doesn't call base class constructors by default (unlike, say, C++ would). However their use may be justified in a situation like yours where you can't modify the API's class -- but you might want to think more about alternatives to do what you need like subclassing or wrapping the API's class instead. Another possibility might be to make an instance of it an attribute of one of your own classes. – Promethium

not sure, wish i could accept both, but yours has been re-accepted :) – Lucilucia

@Ben: Thanks. I can understand your dilemma, but my [biased] opinion is that mine is better because not only does it actually solve your particular problem, it manages to do so in a generic way -- and is thus a little more complicated than just calling the base class's init(). Real world problems can be like that. – Promethium

I really like the mixedomatic decorator. I modified it slightly: classinit = cls.__init__ if '__init__' in cls.__dict__ else None – Kierkegaard

@AlConrad: You don't say what the rationale is for that change—in other words, why you think it's an improved over classinit = getattr(cls, '__init__', None)? – Promethium

@Promethium I don't remember. Was the classinit = getattr(cls, '__init__', None) always there or was it an edit? I think maybe it used to be just classinit = cls.__init__ so I suggested the modification, but using getattr is probably equivalent. I use it here: _mixin_common.py and here optimizers.py – Kierkegaard

@Promethium Actually, now that I think about it I had the following problem: ``` @mixedomatic class SomeCls(SomeMixin, SomeBase): pass ``` SomeCls doesn't have an __init__, but using getattr would produce an init b/c SomeMixin and/or SomeBase might have an init. My suggestion would correctly identify that SomeCls does not have an init. I use that in optimizers.py I mentioned above. – Kierkegaard

@AlConrad: I'm not sure you understand what the code is doing. First of all, classinit = getattr(cls, '__init__', None) produces exactly the same result as what your if statement does, but more succinctly. Secondly, the mixedomatic() decorator always creates an __init__() for the decorated class and the classinit variable is used to determine whether the created method needs to call any __init__() that might have existed already when it gets called (following calls to the ones of all the direct base classes). – Promethium

@Promethium I posted a gist to illustrate. https://gist.github.com/avolkov1/65032f5ec308b05f1bdb0a20814da7f2 Look at lines 56 and 101. See the difference. In the first case you are calling SomeMixin.__init__ twice. That's a bug. – Kierkegaard

@AlConrad: OK, now I see what you mean. In fact getattr() doesn't return the same thing as the if-in statement in the case of a class decorator, since those are executed after a initial unmodified version of the class has already been created—and getattr() follows the MRO of that temporary class and returns the __init__ in the first match in the inheritance hierarchy (the first base class' __init__() here)... – Promethium

...cont This is different from how metaclasses operate because they're a part of single class construction process that occurs using them, so they don't encounter the issue. Thanks for pointing the decorator problem out and suggesting the fix. – Promethium

Thank you. It is great tutorial and new and important things for me. But lets note that the reason of problems in the question is only the missing super() call in APIBaseClassOne. – Astoria

Answer 4

We can learn much from all answers. But I think only Ignacio's answers is CORRECT here.

In other words and very simply said:

If you rewrite some method, always add super() call inside !!! Any method without super() call will break the run of such method for classes/mixins which follows !!!

So the Ignacio's answer is the proper one for the case if WE create the classes.

Problem is that the 3RD PARTY authors write classes which are buggy in this manner. I have found this stackoverflow question while I was working with Django Rest Framework.

class FinalViewSet(viewsets.ModelViewSet, MyMixin)

will fail (the __init__ from MyMixin will not run), while

class FinalViewSet(MyMixin, viewsets.ModelViewSet)

will make things correct.

So lets go to sources of viewsets.ModelViewSet ...

... and we will find that django.views.generic.base.View has incorrect __init__ method.

So the problems is not at DRF authors but at Django authors, in this case.

Astoria

You should probably raise that as an issue, although its possible theres some dark magic going on with that library that necesitates it breaking things. – Catalyst