How to enforce method signature for child classes?

Question

Languages like C#, Java has method overloads, which means if child class does not implement the method with exact signature will not overwrite the parent method.

How do we enforce the method signature in child classes in python? The following code sample shows that child class overwrites the parent method with different method signature:

>>> class A(object):
...   def m(self, p=None):
...     raise NotImplementedError('Not implemented')
... 
>>> class B(A):
...   def m(self, p2=None):
...     print p2
... 
>>> B().m('123')
123

While this is not super important, or maybe by design of python (eg. *args, **kwargs). I am asking this for the sake of clarity if this is possible.

Please Note:

I have tried @abstractmethod and the ABC already.

Collinear

Your example is way more complicated than it should be. The use of abstractmethod has nothing to do with what you're really asking. I strongly recommend remove the use of abstractmethod from your example. – Pitchman

@Pitchman Thanks for the edit, but you should not make large changes to example code in a question, things like indentation is okay, provided it doesn't change the meaning of the code, but larger changes sort of changes the meaning of the question. You may also edit out the actual problem the OP is having by accident. – Shudder

@Carpetsmoker Ok, noted. I guess the best thing is to notify the person asking the question that there is un-necessary cruft in the code? – Pitchman

Yes I agree, I started without using the staticmethod decorator, I put it in just in case someone asks "have you tried the ABC"? – Collinear

@Pitchman Yeah, you can point it out in a comment, or in an answer of course. – Shudder

@JamesLin staticmethod isn't going to help here either. What you're trying to do can probably be accomplished with a custom metaclass. – Pitchman

@JamesLin One more thing. It's really nice if you can post your code in such a way that others can copy/paste it and run it. With all those ... in the code other users have to manually format it before it will run. – Pitchman

Answer 1

Below is a complete running example showing how to use a metaclass to make sure that subclass methods have the same signatures as their base classes. Note the use of the inspect module. The way I'm using it here it makes sure that the signatures are exactly the same, which might not be what you want.

import inspect

class BadSignatureException(Exception):
    pass


class SignatureCheckerMeta(type):
    def __new__(cls, name, baseClasses, d):
        #For each method in d, check to see if any base class already
        #defined a method with that name. If so, make sure the
        #signatures are the same.
        for methodName in d:
            f = d[methodName]
            for baseClass in baseClasses:
                try:
                    fBase = getattr(baseClass, methodName).__func__
                    if not inspect.getargspec(f) == inspect.getargspec(fBase):
                        raise BadSignatureException(str(methodName))
                except AttributeError:
                    #This method was not defined in this base class,
                    #So just go to the next base class.
                    continue

        return type(name, baseClasses, d)


def main():

    class A(object):
        def foo(self, x):
            pass

    try:
        class B(A):
            __metaclass__ = SignatureCheckerMeta
            def foo(self):
                """This override shouldn't work because the signature is wrong"""
                pass
    except BadSignatureException:
        print("Class B can't be constructed because of a bad method signature")
        print("This is as it should be :)")

    try:
        class C(A):
            __metaclass__ = SignatureCheckerMeta
            def foo(self, x):
                """This is ok because the signature matches A.foo"""
                pass
    except BadSignatureException:
        print("Class C couldn't be constructed. Something went wrong")


if __name__ == "__main__":
    main()

Pitchman

Looks like it requires a lot of code to get this done – Collinear

@JamesLin: A lot of code? That metaclass is 12 lines :) – Pitchman

The one problem with this approach is that the enforcing is done by the base class, so this won't stop an unsuspecting user from getting the signature wrong. – Koto

Answer 2

Update of the accepted answer to work with python 3.5.

import inspect
from types import FunctionType

class BadSignatureException(Exception):
    pass


class SignatureCheckerMeta(type):
    def __new__(cls, name, baseClasses, d):
        #For each method in d, check to see if any base class already
        #defined a method with that name. If so, make sure the
        #signatures are the same.
        for methodName in d:
            f = d[methodName]

            if not isinstance(f, FunctionType):
                continue
            for baseClass in baseClasses:
                try:
                    fBase = getattr(baseClass, methodName)
                    if not inspect.getargspec(f) == inspect.getargspec(fBase):
                        raise BadSignatureException(str(methodName))
                except AttributeError:
                    #This method was not defined in this base class,
                    #So just go to the next base class.
                    continue

        return type(name, baseClasses, d)


def main():
    class A(object):
        def foo(self, x):
            pass

    try:
        class B(A, metaclass=SignatureCheckerMeta):
            def foo(self):
                """This override shouldn't work because the signature is wrong"""
                pass
    except BadSignatureException:
        print("Class B can't be constructed because of a bad method signature")
        print("This is as it should be :)")

    try:
        class C(A):
            __metaclass__ = SignatureCheckerMeta
            def foo(self, x):
                """This is ok because the signature matches A.foo"""
                pass
    except BadSignatureException:
        print("Class C couldn't be constructed. Something went wrong")


if __name__ == "__main__":
    main()

Premeditation

Answer 3

mypy, and I expect other static type-checkers, will complain if methods on your subclass have a different signature to the methods they overwrite. It seems to me the best way to enforce type-signatures on child classes is to enforce mypy (or whatever).

Berserker

Answer 4

By design, the language doesn't support checking the signatures. For an interesting read, check out:

http://grokbase.com/t/python/python-ideas/109qtkrzsd/abc-what-about-the-method-arguments

From this thread, it does sound like you may be able to write a decorator to check the signature, with abc.same_signature(method1, method2), but I've never tried that.

Tollhouse

Looks like I misread the thread, alas there is no such method in abc module. Under inspect module, there is getargspec(method), which will show the arg info. You could do a simple comparison of the number of args (and names), if desired. Sorry for not checking it out first! – Tollhouse

You can't type check the arguments, but we do have the inspect module with inspect.getargspec. – Pitchman

Answer 5

The reason it is being overridden is because they actually have the same method signature. What is written there is akin to doing something like this in Java:

public class A
{
    public void m(String p)
    {
        throw new Exception("Not implemented");
    }
}

public class B extends A
{
    public void m(String p2)
    {
        System.out.println(p2);
    }
}

Note that even though the paramater names are different, the types are the same and thus they have the same signature. In strongly typed languages like this, we get to explicitly say what the types are going to be ahead of time.

In python the type of the paramater is dynamically determined at run time when you use the method. This makes it impossible for the python interpreter to tell which method you actually wished to call when you say B().m('123'). Because neither of the method signatures specify which type of paramater they expect, they simply say I'm looking for a call with one parameter. So it makes sense that the deepest (and most relevent to the actual object you have) is called, which would be class B's method because it is an instance of class B.

If you want to only process cetain types in a child class method, and pass along all others to the parent class, it can be done like this:

class A(object):
    def m(self, p=None):
        raise NotImplementedError('Not implemented')

class B(A):
    def m(self, p2=None):
        if isinstance(p2, int):
            print p2
        else:
            super(B, self).m(p2)

Then using b gives you the desired output. That is, class b processes ints, and passes any other type along to its parent class.

>>> b = B()
>>> b.m(2)
2
>>> b.m("hello")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 6, in m
  File "<stdin>", line 3, in m
NotImplementedError: Not implemented

Ladyfinger

Answer 6

I use meta classes for others purposes in my code so I rolled a version that uses a class decorator instead. The below version works with python3. and also supports decorated methods (yes, this creates a potential loophole but if you use decorators that changes the actual signature, shame on you). To make it work with python2, change inspect.isfunction to inspect.ismethod

import inspect
from functools import wraps

class BadSignatureException(Exception):
    pass

def enforce_signatures(cls):
    for method_name, method in inspect.getmembers(cls, predicate=inspect.isfunction):
        if method_name == "__init__":
            continue
        for base_class in inspect.getmro(cls):
            if base_class is cls:
                continue

            try:
                base_method = getattr(base_class, method_name)
            except AttributeError:
                continue

            if not inspect.signature(method) == inspect.signature(base_method):
                raise BadSignatureException("%s.%s does not match base class %s.%s" % (cls.__name__, method_name,
                                                                                       base_class.__name__, method_name))

    return cls

if __name__ == "__main__":
    class A:
        def foo(self, x):
            pass

    def test_decorator(f):
        @wraps(f)
        def decorated_function(*args, **kwargs):
            pass
        return decorated_function

    @enforce_signatures
    class B(A):
        @test_decorator
        def foo(self):
            """This override shouldn't work because the signature is wrong"""
            pass

Premeditation