Enforce specific method signature for subclasses?

Question

I would like to create a class which defines a particular interface, and then require all subclasses to conform to this interface. For example, I would like to define a class

class Interface:
    def __init__(self, arg1):
       pass

    def foo(self, bar):
       pass

and then be assured that if I am holding any element a which has type A, a subclass of Interface, then I can call a.foo(2) it will work.

It looked like this question almost addressed the problem, but in that case it is up to the subclass to explicitly change it's metaclass.

Ideally what I'm looking for is something similar to Traits and Impls from Rust, where I can specify a particular Trait and a list of methods that trait needs to define, and then I can be assured that any object with that Trait has those methods defined.

Is there any way to do this in Python?

Lodged

There probably is a way, but why? Python isn't really the language for this kind of pattern (we're all consenting adults here). – Comparable

@Comparable I suppose you're right. In my case, I'm writing a library where the intention is that the user of the library writes a subclass of my base type and then, as long as they have correctly implemented the methods of the base class (and support the same method signatures) then they can pass that object into the various functions in the library and it will "just work". It would be nicer to have an error show up when it parses their implementation which warns them that "hey, you forgot an argument here", rather than having it show up 6 layers deep in some code or doing something unpredicted. – Lodged

You could follow the pyspark pattern (see my answer for an example) – Comparable

I think the linked question's metaclass based approach would actually work with the base class using the metaclass as well, which would make the child classes use it automatically. – Holoblastic

@Holoblastic That was my original thought, but metaclasses aren't inherited. – Lodged

@process91: Metaclasses are most definitely inherited. If a parent uses a given metaclass, all children of that parent are implicitly of that metaclass. A common pattern to avoid people needing to learn the metaclass syntax is to make a metaclass (e.g. EnumMeta) and a single root class that uses it (e.g. Enum) and have the normal use case for it to be to inherit from the root class (Enum) and get the metaclass automatically without knowing you're doing it. – Holoblastic

@Holoblastic Maybe I don't know how to do it properly, but here's a MWE: repl.it/repls/VigorousKeenApplescript – Lodged

@process91: Your metaclass is broken; it doesn't actually make the new class an instance of the metaclass at all (it calls HelloMeta.__new__, but HelloMeta.__new__ doesn't return an instance of HelloMeta at all, it returns a plain type instance). Change the return line from return type(name, *args) to return super().__new__(cls, name, *args) and it will work properly. – Holoblastic

Thanks! If you wanted to make that an answer, I would accept it :) – Lodged

Answer 1

So, first, just to state the obvious - Python has a built-in mechanism to test for the existence of methods and attributes in derived classes - it just does not check their signature.

Second, a nice package to look at is zope.interface. Despte the zope namespace, it is a complete stand-alone package that allows really neat methods of having objects that can expose multiple interfaces, but just when needed - and then frees-up the namespaces. It sure involve some learning until one gets used to it, but it can be quite powerful and provide very nice patterns for large projects.

It was devised for Python 2, when Python had a lot less features than nowadays - and I think it does not perform automatic interface checking (one have to manually call a method to find-out if a class is compliant) - but automating this call would be easy, nonetheless.

Third, the linked accepted answer at How to enforce method signature for child classes? almost works, and could be good enough with just one change. The problem with that example is that it hardcodes a call to type to create the new class, and do not pass type.__new__ information about the metaclass itself. Replace the line:

return type(name, baseClasses, d)

for:

return super().__new__(cls, name, baseClasses, d)

And then, make the baseclass - the one defining your required methods use the metaclass - it will be inherited normally by any subclasses. (just use Python's 3 syntax for specifying metaclasses).

Sorry - that example is Python 2 - it requires change in another line as well, I better repost it:

from types import FunctionType

# from https://mcmap.net/q/1781051/-how-to-enforce-method-signature-for-child-classes
class SignatureCheckerMeta(type):
    def __new__(mcls, name, baseClasses, d):
        #For each method in d, check to see if any base class already
        #defined a method with that name. If so, make sure the
        #signatures are the same.
        for methodName in d:
            f = d[methodName]
            for baseClass in baseClasses:
                try:
                    fBase = getattr(baseClass, methodName)

                    if not inspect.getargspec(f) == inspect.getargspec(fBase):
                        raise BadSignatureException(str(methodName))
                except AttributeError:
                    #This method was not defined in this base class,
                    #So just go to the next base class.
                    continue

        return super().__new__(mcls, name, baseClasses, d)

On reviewing that, I see that there is no mechanism in it to enforce that a method is actually implemented. I.e. if a method with the same name exists in the derived class, its signature is enforced, but if it does not exist at all in the derived class, the code above won't find out about it (and the method on the superclass will be called - that might be a desired behavior).

The answer:

Fourth - Although that will work, it can be a bit rough - since it does any method that override another method in any superclass will have to conform to its signature. And even compatible signatures would break. Maybe it would be nice to build upon the ABCMeta and @abstractmethod existind mechanisms, as those already work all corner cases. Note however that this example is based on the code above, and check signatures at class creation time, while the abstractclass mechanism in Python makes it check when the class is instantiated. Leaving it untouched will enable you to work with a large class hierarchy, which might keep some abstractmethods in intermediate classes, and just the final, concrete classes have to implement all methods. Just use this instead of ABCMeta as the metaclass for your interface classes, and mark the methods you want to check the interface as @abstractmethod as usual.

class M(ABCMeta):
    def __init__(cls, name, bases, attrs):
        errors = []
        for base_cls in bases:
            for meth_name in getattr(base_cls, "__abstractmethods__", ()):
                orig_argspec = inspect.getfullargspec(getattr(base_cls, meth_name))
                target_argspec = inspect.getfullargspec(getattr(cls, meth_name))
                if orig_argspec != target_argspec:
                    errors.append(f"Abstract method {meth_name!r}  not implemented with correct signature in {cls.__name__!r}. Expected {orig_argspec}.")
        if errors: 
            raise TypeError("\n".join(errors))
        super().__init__(name, bases, attrs)

Rowan

Perfect! I had noticed the same deficiency regarding method existence but didn't know how to integrate @abstractmethod with the metaclass approach. Your final answer checks all the boxes for me, thank you! – Lodged

How does this only have 1 upvote? Really useful and quite non-trivial. – Shel

Answer 2

You could follow the pyspark pattern, where the method of the base class performs (optional) argument validity checking, and then calls a "non-public" method of the subclass, for example:

class Regressor():

    def fit(self, X, y):
        self._check_arguments(X, y)
        self._fit(X, y)

    def _check_arguments(self, X, y):
        if True:
             pass

        else:
            raise ValueError('Invalid arguments.')

class LinearRegressor(Regressor):

    def _fit(self, X, y):
        # code here

Comparable

This is a good solution I feel for most use cases where the developer is just looking to get some simple checks in place for a method signature without getting too deep into Python meta-features. – Outhe

Answer 3

You could follow the pattern common in TypeScript, where you define an interface for the function's parameters.

from dataclasses import dataclass

@dataclass
class MyMethodParams:
    arg1: int
    arg2: str

def mymethod(params: MyMethodParams) -> str:
    return f"{params.arg1} {params.arg2}"

This has the added benefit of simple serialization

import json

from apischema import serialize

params = MyMethodParams(1, "two")
mymethod(params)
json.dump(serialize(params), Path("mymethodparams.json").open("w"), indent=2)

Lattermost