Abstract Enum Class using ABCMeta and EnumMeta [duplicate]

Question

Simple Example

The goal is to create an abstract enum class through a metaclass deriving from both abc.ABCMeta and enum.EnumMeta. For example:

import abc
import enum

class ABCEnumMeta(abc.ABCMeta, enum.EnumMeta):
    pass

class A(abc.ABC):
    @abc.abstractmethod
    def foo(self):
        pass

class B(A, enum.IntEnum, metaclass=ABCEnumMeta):
    X = 1

class C(A):
    pass

Now, on Python3.7, this code will be interpreted without error (on 3.6.x and presumably below, it will not). In fact, everything looks great, our MRO shows B derived from both A and IntEnum.

>>> B.__mro__
(<enum 'B'>, __main__.A, abc.ABC, <enum 'IntEnum'>, int, <enum 'Enum'>, object)

Abstract Enum is not Abstract

However, even though B.foo has not been defined, we can still instantiate B without any issue, and call foo().

>>> B.X
<B.X: 1>
>>> B(1)
<B.X: 1>
>>> B(1).foo() 

This seems rather weird, since any other class that derives from ABCMeta cannot be instantiated, even if I use a custom metaclass.

>>> class NewMeta(type): 
...     pass
... 
... class AbcNewMeta(abc.ABCMeta, NewMeta):
...     pass
... 
... class D(metaclass=NewMeta):
...     pass
... 
... class E(A, D, metaclass=AbcNewMeta):
...     pass
...
>>> E()
TypeError: Can't instantiate abstract class E with abstract methods foo

Question

Why does a class using a metaclass derived from EnumMeta and ABCMeta effectively ignore ABCMeta, while any other class using a metaclass derived from ABCMeta use it? This is true even if I custom define the __new__ operator.

class ABCEnumMeta(abc.ABCMeta, enum.EnumMeta):
    def __new__(cls, name, bases, dct):
        # Commented out lines reflect other variants that don't work
        #return abc.ABCMeta.__new__(cls, name, bases, dct)
        #return enum.EnumMeta.__new__(cls, name, bases, dct)
        return super().__new__(cls, name, bases, dct)

I'm rather confused, since this seems to fly in the face of what a metaclass is: the metaclass should define how the class is defined and behaves, and in this case, defining a class using a metaclass that is both abstract and an enumeration creates a class that silently ignores the abstract component. Is this a bug, is this intended, or is there something greater I am not understanding?

Follett

B(1) does not create an instance of B. – Godber

@Godber Mark that as answer, since digging deeper through your comments I was able to find this: github.com/python/cpython/blob/… Write it up as an answer and I'll mark it as answered. Thank you. Didn't realize that all class construction was done by the metaclass, which makes obvious sense in retrospect. – Follett

I'm not quite confident enough to do that; I've been digging through enum.py to see exactly what EnumMeta does. – Godber

You shall not create AbstractEnum normally, because Enum members are created at the moment of Enum class creation (and then they cannot be added later on). It make sense though to do so if you use aenum package which provides tool (extend_enum function) to create enum members dynamically. Unfortunately, I have no idea how to achieve this goal :( – Lancers

Answer 1

As stated on @chepner's answer, what is going on is that Enum metaclass overrides the normal metaclass' __call__ method, so that an Enum class is never instantiated through the normal methods, and thus, ABCMeta checking does not trigger its abstractmethod check.

However, on class creation, the Metaclass's __new__ is run normally, and the attributes used by the abstract-class mechanisms to mark the class as abstract do create the attribute ___abstractmethods__ on the created class.

So, all you have to do for what you intend to work, is to further customize your metaclass to perform the abstract check in the code to __call__:

import abc
import enum

class ABCEnumMeta(abc.ABCMeta, enum.EnumMeta):

    def __call__(cls, *args, **kw):
        if getattr(cls, "__abstractmethods__", None):
            raise TypeError(f"Can't instantiate abstract class {cls.__name__} "
                            f"with frozen methods {set(cls.__abstractmethods__)}")
        return super().__call__(*args, **kw)

This will make the B(1) expression to fail with the same error as abstractclass instantiation.

Note, however, that an Enum class can't be further inherited anyway, and it simply creating it without the missing abstractmethods may already violate what you want to check. That is: in your example above, class B can be declared and B.x will work, even with the missing foo method. If you want to prevent that, just put the same check in the metaclass' __new__:

import abc
import enum

class ABCEnumMeta(abc.ABCMeta, enum.EnumMeta):

    def __new__(mcls, *args, **kw):
        cls = super().__new__(mcls, *args, **kw)
        if issubclass(cls, enum.Enum) and getattr(cls, "__abstractmethods__", None):
            raise TypeError("...")
        return cls

    def __call__(cls, *args, **kw):
        if getattr(cls, "__abstractmethods__", None):
            raise TypeError(f"Can't instantiate abstract class {cls.__name__} "
                            f"with frozen methods {set(cls.__abstractmethods__)}")
        return super().__call__(*args, **kw)

(Unfortunatelly, the ABC abstract method check in CPython seems to be performed in native code, outside the ABCMeta.__call__ method - otherwise, instead of mimicking the error, we could just call ABCMeta.__call__ explicitly overriding super's behavior instead of hardcoding the TypeError there.)

Eros

This doesn't work (at least on 3.6). – Erlond

Answer 2

Calling an enumerated type doesn't create a new instance. Members of the enumerated type are created immediately at class-creation time by the meta class. The __new__ method simply performs lookup, which means ABCMeta is never invoked to prevent instantiation.

B(1).foo() works because, once you have an instance, it doesn't matter if the method was marked as abstract. It's still a real method, and can be called as such.

Godber