python abstractmethod with another baseclass breaks abstract functionality

Question

Consider the following code example

import abc
class ABCtest(abc.ABC):
    @abc.abstractmethod
    def foo(self):
        raise RuntimeError("Abstract method was called, this should be impossible")

class ABCtest_B(ABCtest):
    pass

test = ABCtest_B()

This correctly raises the error:

Traceback (most recent call last):
  File "/.../test.py", line 10, in <module>
    test = ABCtest_B()
TypeError: Can't instantiate abstract class ABCtest_B with abstract methods foo

However when the subclass of ABCtest also inherits from a built in type like str or list there is no error and test.foo() calls the abstract method:

class ABCtest_C(ABCtest, str):
    pass

>>> test = ABCtest_C()
>>> test.foo()
Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    test.foo()
  File "/.../test.py", line 5, in foo
    raise RuntimeError("Abstract method was called, this should be impossible")
RuntimeError: Abstract method was called, this should be impossible

This seems to happen when inheriting from any class defined in C including itertools.chain and numpy.ndarray but still correctly raises errors with classes defined in python. Why would implementing one of a built in types break the functionality of abstract classes?

Sabol

@DonkeyKong (or anyone else who doesn't get it) the method foo should be enforced to be overridden in a subclass, normally (and without also inheritting from str) instantiating it raises an error, however when also inherriting from str no error happens and the abstract method test.foo is a valid callable method. – Aspectual

@TadhgMcDonald-Jensen Just caught on, thanks :) – Fuqua

@Torxed str isn't a variable name. – Fuqua

I just realized, thanks for making me read it twice @DonkeyKong :) – Trass

considering the error that is normally raised is definitely not raised from abc.py since there is no entry in the traceback I think that the error originates from type.__call__ so I think a look into the C source code will be required to answer why this is happening... – Aspectual

oh heck - well if anyone is into the source, I'd be delighted. But is this part of a specification? could there be a situation where this behaviour is desired? or is it a bug, or simply case that has not been regarded? – Sabol

Answer 1

Surprisingly, the test that prevents instantiating abstract classes happens in object.__new__, rather than anything defined by the abc module itself:

static PyObject *
object_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
    ...
    if (type->tp_flags & Py_TPFLAGS_IS_ABSTRACT) {
        ...
        PyErr_Format(PyExc_TypeError,
                     "Can't instantiate abstract class %s "
                     "with abstract methods %U",
                     type->tp_name,
                     joined);

(Almost?) all built-in types that aren't object supply a different __new__ that overrides object.__new__ and does not call object.__new__. When you multiple-inherit from a non-object built-in type, you inherit its __new__ method, bypassing the abstract method check.

I don't see anything about __new__ or multiple inheritance from built-in types in the abc documentation. The documentation could use enhancement here.

It seems kind of strange that they'd use a metaclass for the ABC implementation, making it a mess to use other metaclasses with abstract classes, and then put the crucial check in core language code that has nothing to do with abc and runs for both abstract and non-abstract classes.

There's a report for this issue on the issue tracker that's been languishing since 2009.

Pedroza

Answer 2

I asked a similar question and based on user2357112 supports Monicas linked bug report, I came up with this workaround (based on the suggestion from Xiang Zhang):

from abc import ABC, abstractmethod

class Base(ABC):
    @abstractmethod
    def foo(self):
        pass

    @abstractmethod
    def bar(self):
        pass

    def __new__(cls, *args, **kwargs):
        abstractmethods = getattr(cls, '__abstractmethods__', None)
        if abstractmethods:
            msg = "Can't instantiate abstract class {name} with abstract method{suffix} {methods}"
            suffix = 's' if len(abstractmethods) > 1 else ''
            raise TypeError(msg.format(name=cls.__name__, suffix=suffix, methods=', '.join(abstractmethods)))
        return super().__new__(cls, *args, **kwargs)

class Derived(Base, tuple):
    pass

Derived()

This raises TypeError: Can't instantiate abstract class Derived with abstract methods bar, foo, which is the original behaviour.

Goliath