Triple inheritance causes metaclass conflict... Sometimes

Question

Looks like I stumbled upon a metaclass hell even when I didn't wanted anything to do with it.

I'm writing an app in Qt4 using PySide. I want to separate event-driven part from UI definition, which is generated from Qt Designer files. Hence I create a "controller" classes, but to ease my life I multiple-inherit them anyways. An example:

class BaseController(QObject):
    def setupEvents(self, parent):
        self.window = parent

class MainController(BaseController):
    pass

class MainWindow(QMainWindow, Ui_MainWindow, MainController):
    def __init__(self, parent=None):
        super(MainWindow, self).__init__(parent)

        self.setupUi(self)
        self.setupEvents(self)

This works as expected. It also has inheritance from (QDialog, Ui_Dialog, BaseController). But when I subclass BaseController and try to inherit from said subclass (in place of BaseController), I receive an error:

TypeError: Error when calling the metaclass bases metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

Clarification: Both QMainWindow and QDialog inherit from QObject. BaseController must also inherit from it because of Qt event system peculiarities. Ui_ classes only inherit from simple Python object class. I searched for solutions, but all of them involve cases of intentionally using metaclasses. So I must be doing something terribly wrong.

EDIT: My description may be clearer by adding graphs.

Working example:

QObject
|      \___________________
|            object        |
QMainWindow     |          BaseController
|      /---Ui_MainWindow   |
|      |                   MainController
MainWindow-----------------/

Another working example:

QObject
|      \___________________
|            object        |
QDialog         |          BaseController
|      /---Ui_OtherWindow  |
|      |                   |
OtherWindow----------------/

Not working example:

QObject
|      \___________________
|            object        |
QDialog         |          BaseController
|      /---Ui_OtherWindow  |
|      |                   OtherController
OtherWindow----------------/

Palatal

I'm not very good at Python metaclasses, but I think the problem may be with ordering of parent classes within your MainWindow class definition. Just a guess. – Michele

What most confuses me is that MainWindow works, while putting QDialog, Ui_Dialog and controller in the same sequence: class inheriting from QObject, class inheriting from object, class inheriting from QObject - for some reason fails. – Palatal

I don't know about python, but in C++/Qt, multiple inheritance from QObject is strictly disallowed. I wonder if you are encountering the same problem, and it just happens to work in some of your cases. – Beetner

From your descriptions, I could not reproduce the issue you are describing. Here is a paste of the code: pastie.org/2287381 – Ljoka

If you have problems with multiple inheritance, then as a fallback you can always use single inheritance and 3 different objects having references to each other. – Hammons

what python version are you using? – Heurlin

Man, this is one of the scariest question titles I have seen here! – Novokuznetsk

Answer 1

The error message indicates that you have two conflicting metaclasses somewhere in your hierarchy. You need to examine each of your classes and the QT classes to figure out where the conflict is.

Here's some simple example code that sets up the same situation:

class MetaA(type):
    pass
class MetaB(type):
    pass
class A:
    __metaclass__ = MetaA
class B:
    __metaclass__ = MetaB

We can't subclass both of those classes directly, because python wouldn't know which metaclass to use:

>>> class Broken(A, B): pass
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Error when calling the metaclass bases
  metaclass conflict: the metaclass of a derived class must be a (non-strict)
  subclass of the metaclasses of all its bases

What the error is trying to tell us is that we need to resolve the conflict between the two metaclasses by introducing a third metaclass that is a subclass of all the metaclasses from the base classes.

I'm not sure that's any clearer than the error message itself, but basically, you fix it by doing this:

class MetaAB(MetaA, MetaB):
    pass

class Fixed(A, B):
    __metaclass__ = MetaAB

This code now compiles and runs correctly. Of course, in the real situation, your conflict-resolving metaclass would have to decide which of the parent metaclass behaviors to adopt, which you'll have to figure out for yourself from your application's requirements.

Bear in mind that your inherited class only gets one of the two metaclass.__init__ methods, which sometimes do all the work, so in a lot of cases, you are going to have to add an __init__ that calls into both in some way that helps them get along.

Understanding

Definetly correct answer, thanks for helping me understand that. I've learned a bit about programming methodologies (i.e. adapting object rather than inheriting it's class...) but it is still usable to have this knowledge when i deal with metaclasses where i need them. Thanks again. – Palatal

What does the parenthesized adjective non-strict signify? When is a subclass considered strict and when not? – Bosk

Answer 2

We use something like this:

class CooperativeMeta(type):
    def __new__(cls, name, bases, members):
        #collect up the metaclasses
        metas = [type(base) for base in bases]

        # prune repeated or conflicting entries
        metas = [meta for index, meta in enumerate(metas)
            if not [later for later in metas[index+1:]
                if issubclass(later, meta)]]

        # whip up the actual combined meta class derive off all of these
        meta = type(name, tuple(metas), dict(combined_metas = metas))

        # make the actual object
        return meta(name, bases, members)

    def __init__(self, name, bases, members):
        for meta in self.combined_metas:
            meta.__init__(self, name, bases, members)

Assuming good, modern metaclass-implementation hygiene (where metaclasses subclass type, and whatever can be done in __init__ gets done there) this allows many metaclasses to get along.

Metaclasses that really and necessarily do most of their work in __new__, are going to be hard to combine anyway. You can sneak one of them in here by making sure its class is the first element in the multiple inheritance.

To use this, you just declare:

__metaclass__ = CooperativeMeta

for those classes where different metaclasses come together.

In this case, for instance:

class A:
    __metaclass__ = MetaA
class B:
    __metaclass__ = MetaB
class Fixed(A, B):
    __metaclass__ = CooperativeMeta

This is many times more likely to work correctly across the board for different MetaA and MetaB than just inheriting them together to shut the compiler up.

Hopefully the comments explain the code. There is just one tricky line, and it is about removing redundant calls to any given __metaclass__ inherited from different places and allowing classes with no explicit metaclass to play nicely with the others. If it seems overreaching, you can omit it and in your code, just order the base classes carefully.

That makes the solution three lines and pretty clear.

Ontogeny