“Can't instantiate abstract class … with abstract methods” on class that shouldn't have any abstract method

Question

Take the following minimal example:

import abc

class FooClass(object):
  __metaclass__ = abc.ABCMeta

  @abc.abstractmethod
  def FooMethod(self):
    raise NotImplementedError()


def main():
  derived_type = type('Derived', (FooClass,), {})

  def BarOverride(self):
    print 'Hello, world!'
  derived_type.FooMethod = BarOverride

  instance = derived_type()

Running main() gets you:

TypeError: Can't instantiate abstract class Derived with abstract methods FooMethod

(The exception occurs on the instance = derived_type() line.)

But FooMethod shouldn't be abstract: I've overridden it with BarOverride. So, why is this raising exceptions?

Disclaimer: Yes, I could use the explicit class syntax, and accomplish the exact same thing. (And even better, I can make it work!) But this is a minimal test case, and the larger example is dynamically creating classes. :-) And I'm curious as to why this doesn't work.

Edit: And to prevent the other obvious non-answer: I don't want to pass BarOverride in the third argument to type: In the real example, BarOverride needs to have derived_type bound to it. It is easier to do this if I can define BarOverride after the creation of derived_type. (If I can't do this, then why?)

Delphina

The abstractness of a class is determined during class construction. Why don't you simply include FooMethod in the dictionary while creating the class? – Sorry

@SvenMarnach: BarOverride, in the real example, needs to have the derived class bound to it. Creating the function afterwards is the easiest way to accomplish this. – Delphina

I don't get that point. What do you mean by "needs to have the derived class bound to it"? – Sorry

Answer 1

Because the docs say so:

Dynamically adding abstract methods to a class, or attempting to modify the abstraction status of a method or class once it is created, are not supported. The abstractmethod() only affects subclasses derived using regular inheritance; “virtual subclasses” registered with the ABC’s register() method are not affected.

A metaclass is only called when a class is defined. When abstractmethod has marked a class as abstract that status won't change later.

Cognizance

Accepting this as the answer (because it contains the reason my code doesn't work), but all three are interesting. I've selected unutbu's answer as the workaround to the problem at hand. – Delphina

Answer 2

Jochen is right; the abstract methods are set at class creation and won't me modified just because you reassign an attribute.

You can manually remove it from the list of abstract methods by doing

DerivedType.__abstractmethods__ = frozenset()

or

DerivedType.__abstractmethods__ = frozenset(
        elem for elem in DerivedType.__abstractmethods__ if elem != 'FooMethod')

as well as setattr, so it doesn't still think that FooMethod is abstract.

Ignacia

Answer 3

I know this topic is really old but... That is really a nice question.

It doesn't work because abc can only check for abstract methods during instatiation of types, that is, when type('Derived', (FooClass,), {}) is running. Any setattr done after that is not accessible from abc.

So, setattr wont work, buuut... Your problem of addressing the name of a class that wasn't previously declared or defined looks solvable:

I wrote a little metaclass that lets you use a placeholder "clazz" for accessing any class that will eventually get the method you are writing outside a class definition.

That way you won't get TypeError from abc anymore, since you can now define your method BEFORE instatiating your type, and then pass it to type at the dict argument. Then abc will see it as a proper method override.

Aaand, with the new metaclass you can refer to the class object during that method. And this is super, because now you can use super! =P I can guess you were worried about that too...

Take a look:

import abc
import inspect

clazz = type('clazz', (object,), {})()

def clazzRef(func_obj):
    func_obj.__hasclazzref__ = True
    return func_obj

class MetaClazzRef(type):
    """Makes the clazz placeholder work.

    Checks which of your functions or methods use the decorator clazzRef
    and swaps its global reference so that "clazz" resolves to the
    desired class, that is, the one where the method is set or defined.

    """
    methods = {}
    def __new__(mcs, name, bases, dict):
        ret = super(MetaClazzRef, mcs).__new__(mcs, name, bases, dict)
        for (k,f) in dict.items():
            if getattr(f, '__hasclazzref__', False):
                if inspect.ismethod(f):
                    f = f.im_func
                if inspect.isfunction(f):
                    for (var,value) in f.func_globals.items():
                        if value is clazz:
                            f.func_globals[var] = ret
        return ret

class MetaMix(abc.ABCMeta, MetaClazzRef):
    pass

class FooClass(object):
    __metaclass__ = MetaMix
    @abc.abstractmethod
    def FooMethod(self):
        print 'Ooops...'
        #raise NotImplementedError()


def main():
    @clazzRef
    def BarOverride(self):
        print "Hello, world! I'm a %s but this method is from class %s!" % (type(self), clazz)
        super(clazz, self).FooMethod() # Now I have SUPER!!!

    derived_type = type('Derived', (FooClass,), {'FooMethod': BarOverride})

    instance = derived_type()
    instance.FooMethod()

    class derivedDerived(derived_type):
        def FooMethod(self):
            print 'I inherit from derived.'
            super(derivedDerived,self).FooMethod()

    instance = derivedDerived()
    instance.FooMethod()

main()

The output is:

Hello, world! I'm a <class 'clazz.Derived'> but this method is from class <class 'clazz.Derived'>!
Ooops...
I inherit from derived.
Hello, world! I'm a <class 'clazz.derivedDerived'> but this method is from class <class 'clazz.Derived'>!
Ooops...

Constantina

I guess this is wrong. I should be doing this with function's closures, not globals(). Globals() are, err, global... But, strangely enough, the code looks to work as expected, even with more classes... – Constantina

Answer 4

Well, if you must do it this way, then you could just pass a dummy dict {'FooMethod':None} as the third argument to type. This allows derived_type to satisfy ABCMeta's requirement that all abstract methods be overridden. Later on you can supply the real FooMethod:

def main():
  derived_type = type('Derived', (FooClass,), {'FooMethod':None})
  def BarOverride(self):
    print 'Hello, world!'
  setattr(derived_type, 'FooMethod', BarOverride)
  instance = derived_type()

Dourine

But this defeats some of the point of making FooMethod abstract, because if the type creation and setattr are separated, you could instantiate derived_type before setting the method. – Ignacia

A class of type ABCMeta should be monkey-patchable just like any other class. By supplying the third argument you acknowledge that Derived must define FooMethod. That does not bar you from changing your mind about its implementation later. – Dourine