Is there a way to persist decorators during inheritance?

Question

I'm trying to write an abstract class with unimplemented methods, which will force the inheriting children to return a value of a specific type when they override the method (defined in the decorator).

When I use the code shown below, the child method does not call the decorator. I assume this is because the method is being overridden, which makes a lot of sense. My question is basically this: Is there a way to make a decorator persist through method overriding?

I'm not opposed to using something other than a decorator, but this was a solution that quickly popped to mind and I'm curios to know if there's any way to make it work.

In case going with a decorator is the correct and possible choice, it will look something like this:

def decorator(returntype):
    def real_decorator(function):
        def wrapper(*args, **kwargs):
            result = function(*args, **kwargs)
            if not type(result) == returntype:
                raise TypeError("Method must return {0}".format(returntype))
            else:
                return result
        return wrapper
    return real_decorator

I need my parent class to look similar to this one:

class Parent(ABC):
    @decorator(int)
    @abstractmethod
    def aye(self, a):
        raise NotImplementedError

And the child class will do something of this sort:

class Child(Parent):
    def aye(self, a):
        return a

I'd be more than happy to clarify my question better if needed, and thank you to everyone who takes the time to read this question in advance!

Sheffield

Yes. For example with a metaclass where you change the selected method to have the decorator applied. See here and here and here and here for good ideas and samples. – Callow

You should also metathink if you really have good use cases for doing that. – Callow

You are also asking to do two different things. As @mgc mentions maybe static type checking is enough. Forcing signatures seems always anti-python to me but it may be needed/useful in certain cases. Not questioning that. – Callow

Anyway I found your question really interesting. Hence the several research links. :) – Callow

if return type enforcement's your only goal, could you not use something simpler? say def aye(self, a): res = self._aye(a) assert isinstance(res, int) return res where _aye is the actual method on subclasses and aye is its public, type-checking invocator? – Expanse

Thank you for taking interest in my question @progmatico! I must make sure these methods fail with the wrong return type because of certain restrictions with the consumers of this class. @mgc 's solution could work since I might be able to make the CI/CD process fail if mypy doesn't pass, but I absolutely must enforce the return type :) – Sheffield

@JLPeyret that is a very viable solution and could work in my use case. I'll have to run it against other people on my team, but thank you very very much for your answer! – Sheffield

@JLPeyret in my opinion your approach (which also clearly involves less code than the decorator proposed by OP) is bit less reusable and a bit less "obvious" than decorating the function (and, as in my first snippet below, it still create some indirection for the developers when subclassing the Parent class as you have to implement a method with an other name than the actual method that is called). Well, I'm nitpicking, it also does the job in the end ! – Mapping

@Sheffield see my comment on asserting in the JL Peyret answer. – Callow

Answer 1

I'm not sure you can persists the effect of the decorator the way you want to, but you can still decorate a wrapper function in the Parent class which will not be an abstractmethod and let the children class implement the wrapped function like that :

from abc import ABC, abstractmethod

def decorator(returntype):
    def real_decorator(function):
        def wrapper(*args, **kwargs):
            result = function(*args, **kwargs)
            if not type(result) == returntype:
                raise TypeError("Method must return {0}".format(returntype))
            else:
                return result
        return wrapper
    return real_decorator

class Parent(ABC):
    @decorator(int)
    def aye(self, a):
        return self.impl_aye(a)

    @abstractmethod
    def impl_aye(self, a):
        raise NotImplementedError


class Child(Parent):
    def impl_aye(self, a):
        return a

There is also solutions to protect the aye method from the Parent class to be overridden if you need it, see this answer for example.

Otherwise if you want to use type hints and check your code with mypy (an optional static type checker for Python) you can get error message if you try to implement a child class with a return type incompatible with its parent class :

from abc import ABC, abstractmethod

class Parent(ABC):
    @abstractmethod
    def aye(self, a) -> int:
        raise NotImplementedError

class Child(Parent):
    def aye(self, a) -> str :
        return a

Output of mypy:

a.py:9: error: Return type "str" of "aye" incompatible with return type "int" in supertype "Parent"
Found 1 error in 1 file (checked 1 source file)

Mapping

That is a great possible solution, if I can use it to make my CI/CD process fail I might choose this answer. Thank you very much! – Sheffield

You're welcome ! If I understand well your problem, I guess you can make your CI process to fail with both of my proposition (although I'm not very proud of my first proposition, as it create an indirection for the developers, as you have to implement impl_aye (or any other name) when the caller is calling aye, so don't forget there is also good suggestions in @progrmatico first comment !) – Mapping

Answer 2

IF you only want to enforce return type, here's my non-decorator suggestion (hadn't originally put it in as I am not fond of you-dont-want-to-do-this "answers" on SO).

class Parent:

    def aye(self, a):
        res = self._aye(a)
        if not isinstance(res, int):
            raise TypeError("result should be an int")
        return res

    def _aye(self, a):
        raise NotImplementedError()

class Child(Parent):

    def _aye(self, a):
        return 1

Expanse

Would be even better to raise an exception instead of asserting, as asserts can be disabled. – Callow

agreed and changed. – Expanse

Answer 3

Here's how you could do this with a metaclass. Tested on Python 3.8. Should work as-is on 3.6 and up. Admittedly, this is somewhat complex, and it might be better to use another technique.

from abc import ABCMeta, abstractmethod
from functools import wraps
from inspect import isfunction


class InheritedDecoratorMeta(ABCMeta):

    def __init__(cls, name, bases, attrs):
        for name, attr in attrs.items():
            for base in bases:
                base_attr = base.__dict__.get(name)
                if isfunction(base_attr):
                    inherited_decorator = getattr(base_attr, 'inherited_decorator', None)
                    if inherited_decorator:
                        setattr(cls, name, inherited_decorator()(attr))
                        break


def inherited_decorator(decorator, result_callback):
    def inner_decorator(method):
        method.inherited_decorator = lambda: inherited_decorator(decorator, result_callback)
        @wraps(method)
        def wrapper(*args, **kwargs):
            result = method(*args, **kwargs)
            return result_callback(method, result, args, kwargs)
        return wrapper
    return inner_decorator


def returns(type_):
    if not isinstance(type_, type) and type_ is not None:
        raise TypeError(f'Expected type or None; got {type_}')
    def callback(method, result, args, kwargs):
        result_type = type(result)
        if type_ is None:
            if result is not None:
                raise TypeError(f'Expected method {method} to return None; got {result_type}')
        elif not isinstance(result, type_):
            raise TypeError(f'Expected method {method} to return {type_}; got {result_type}')
        return result
    return inherited_decorator(returns, callback)


class MyBaseClass(metaclass=InheritedDecoratorMeta):

    @returns(int)
    @abstractmethod
    def aye(self, a):
        raise NotImplementedError

    @returns(None)
    @abstractmethod
    def bye(self, b):
        raise NotImplementedError


class MyClass(MyBaseClass):

    def aye(self, a):
        return a

    def bye(self, b):
        return b

    @returns(str)
    def cye(self, c):
        return c


if __name__ == '__main__':
    instance = MyClass()

    instance.aye(1)
    try:
        instance.aye('1')
    except TypeError as exc:
        print(exc)

    instance.bye(None)
    try:
        instance.bye(1)
    except TypeError as exc:
        print(exc)

    instance.cye('string')
    try:
        instance.cye(1)
    except TypeError as exc:
        print(exc)

Epochmaking

Answer 4

One way to keep the overwritten abstractmethods wrapped by decorators, is to define what happens when the superclass is subclassed.

So, to maintain the decorators we can wrap the overwritten methods in __init_subclass__ method of the superclass.

In your example, we can do this by adding just three lines of code to the superclass:

def __init_subclass__(cls) -> None:
    super().__init_subclass__()
    # Get the decorator, and wrap the 'aye' function to enforce 'int' type
    cls.aye = decorator(int)(cls.aye)

This gives us the desired effect even in subclasses:

c = Child()
print(c.aye(1))     # 1
print(c.aye("s"))   # TypeError: Method must return <class 'int'>

Auxiliaries