Python class decorator arguments

Question

I'm trying to pass optional arguments to my class decorator in python. Below the code I currently have:

class Cache(object):
    def __init__(self, function, max_hits=10, timeout=5):
        self.function = function
        self.max_hits = max_hits
        self.timeout = timeout
        self.cache = {}

    def __call__(self, *args):
        # Here the code returning the correct thing.


@Cache
def double(x):
    return x * 2

@Cache(max_hits=100, timeout=50)
def double(x):
    return x * 2

The second decorator with arguments to overwrite the default one (max_hits=10, timeout=5 in my __init__ function), is not working and I got the exception TypeError: __init__() takes at least 2 arguments (3 given). I tried many solutions and read articles about it, but here I still can't make it work.

Any idea to resolve this? Thanks!

Yardarm

Answer 1

@Cache(max_hits=100, timeout=50) calls __init__(max_hits=100, timeout=50), so you aren't satisfying the function argument.

You could implement your decorator via a wrapper method that detected whether a function was present. If it finds a function, it can return the Cache object. Otherwise, it can return a wrapper function that will be used as the decorator.

class _Cache(object):
    def __init__(self, function, max_hits=10, timeout=5):
        self.function = function
        self.max_hits = max_hits
        self.timeout = timeout
        self.cache = {}

    def __call__(self, *args):
        # Here the code returning the correct thing.

# wrap _Cache to allow for deferred calling
def Cache(function=None, max_hits=10, timeout=5):
    if function:
        return _Cache(function)
    else:
        def wrapper(function):
            return _Cache(function, max_hits, timeout)

        return wrapper

@Cache
def double(x):
    return x * 2

@Cache(max_hits=100, timeout=50)
def double(x):
    return x * 2

Spoilt

If the developer callsCache with positional instead of keyword arguments (e.g. @Cache(100,50)) then function will be assigned the value 100, and max_hits 50. An error won't be raised until the function is called. This could be considered surprising behavior since most people expect uniform positional and keyword semantics. – Martell

If i use the @Cache decorator on an object instance method, then the __call__ method of _Cache doesn't receive the self reference of the decorated object. Doesn't work in this case. – Nazareth

wow. this works, on regular functions – Serenity

to fix problem mentioned by @Martell we can define decorator in a following way: def Cache(function=None, *, max_hits=10, timeout=5): – Norrisnorrv

Answer 2

@Cache
def double(...): 
   ...

is equivalent to

def double(...):
   ...
double=Cache(double)

While

@Cache(max_hits=100, timeout=50)
def double(...):
   ...

is equivalent to

def double(...):
    ...
double = Cache(max_hits=100, timeout=50)(double)

Cache(max_hits=100, timeout=50)(double) has very different semantics than Cache(double).

It's unwise to try to make Cache handle both use cases.

You could instead use a decorator factory that can take optional max_hits and timeout arguments, and returns a decorator:

class Cache(object):
    def __init__(self, function, max_hits=10, timeout=5):
        self.function = function
        self.max_hits = max_hits
        self.timeout = timeout
        self.cache = {}

    def __call__(self, *args):
        # Here the code returning the correct thing.

def cache_hits(max_hits=10, timeout=5):
    def _cache(function):
        return Cache(function,max_hits,timeout)
    return _cache

@cache_hits()
def double(x):
    return x * 2

@cache_hits(max_hits=100, timeout=50)
def double(x):
    return x * 2

PS. If the class Cache has no other methods besides __init__ and __call__, you can probably move all the code inside the _cache function and eliminate Cache altogether.

Martell

unwise or not... if the developer does accidentally use @cache instead of cache(), it'll make a weird error when they try to call the resulting function. the other implementation actually works as both cache and cache() – Spoilt

@lunixbochs: A developer who confuses cache_hits (nee cache) with cache_hits() is just as likely to confuse any function object with a function call, or mistake a generator with an iterator. Even moderately experienced Python programmers should be used to paying attention to the differenc. – Martell

Answer 3

I'd rather to include the wrapper inside the class's __call__ method:

UPDATE: This method has been tested in python 3.6, so I'm not sure about the higher or earlier versions.

class Cache:
    def __init__(self, max_hits=10, timeout=5):
        # Remove function from here and add it to the __call__
        self.max_hits = max_hits
        self.timeout = timeout
        self.cache = {}

    def __call__(self, function):
        def wrapper(*args):
            value = function(*args)
            # saving to cache codes
            return value
        return wrapper

@Cache()
def double(x):
    return x * 2

@Cache(max_hits=100, timeout=50)
def double(x):
    return x * 2

Large

Have you tried calling the functions after decorating them? I think this method doesn't work. – Rentfree

@AK12 Have you tried it or you just think it wont work? Cz I'm working with this method and it works fine. – Large

I've tried and got errors. The error happens when I try to call the double method. – Rentfree

@AK12 What error do you get? Cz I just tried it in a test project with Python3.6 and it works – Large

@AK12 Did you try copy pasting the whole answer in a new project and run for example double(5)? What's you python version? – Large

I use python 3.8, I copy-pasted the call method and the two decorated methods. – Rentfree

@AK12 If you have copied the whole class and the problem still exists, then I suspect the issue could be cause of our differenet Python versions. – Large

Just a note: This class is a decorator factory, while the __call__ function defines the decorator itself. IMO this is much cleaner then the accepted answer. You could also add @functools.wraps(function) infront of your wrapper function in order to improve your answer even further. – Inceptive

This works on Python 3.10.13. – Dermatitis

Answer 4

I've learned a lot from this question, thanks all. Isn't the answer just to put empty brackets on the first @Cache? Then you can move the function parameter to __call__.

class Cache(object):
    def __init__(self, max_hits=10, timeout=5):
        self.max_hits = max_hits
        self.timeout = timeout
        self.cache = {}

    def __call__(self, function, *args):
        # Here the code returning the correct thing.

@Cache()
def double(x):
    return x * 2

@Cache(max_hits=100, timeout=50)
def double(x):
    return x * 2

Although I think this approach is simpler and more concise:

def cache(max_hits=10, timeout=5):
    def caching_decorator(fn):
        def decorated_fn(*args ,**kwargs):
            # Here the code returning the correct thing.
        return decorated_fn
    return decorator

If you forget the parentheses when using the decorator, unfortunately you still don't get an error until runtime, as the outer decorator parameters are passed the function you're trying to decorate. Then at runtime the inner decorator complains:

TypeError: caching_decorator() takes exactly 1 argument (0 given).

However you can catch this, if you know your decorator's parameters are never going to be a callable:

def cache(max_hits=10, timeout=5):
    assert not callable(max_hits), "@cache passed a callable - did you forget to parenthesize?"
    def caching_decorator(fn):
        def decorated_fn(*args ,**kwargs):
            # Here the code returning the correct thing.
        return decorated_fn
    return decorator

If you now try:

@cache
def some_method()
    pass

You get an AssertionError on declaration.

On a total tangent, I came across this post looking for decorators that decorate classes, rather than classes that decorate. In case anyone else does too, this question is useful.

Kelwunn

Answer 5

You can use a classmethod as a factory method, this should handle all the use cases (with or without parenthesis).

import functools
class Cache():
    def __init__(self, function):
        functools.update_wrapper(self, function)
        self.function = function
        self.max_hits = self.__class__.max_hits
        self.timeout = self.__class__.timeout
        self.cache = {}

    def __call__(self, *args):
        # Here the code returning the correct thing.
    
    @classmethod
    def Cache_dec(cls, _func = None, *, max_hits=10, timeout=5):
        cls.max_hits = max_hits
        cls.timeout = timeout
        if _func is not None: #when decorator is passed parenthesis
            return cls(_func)
        else:
            return cls    #when decorator is passed without parenthesis
       

@Cache.Cache_dec
def double(x):
    return x * 2

@Cache.Cache_dec()
def double(x):
    return x * 2

@Cache.Cache_dec(timeout=50)
def double(x):
    return x * 2

@Cache.Cache_dec(max_hits=100)
def double(x):
    return x * 2

@Cache.Cache_dec(max_hits=100, timeout=50)
def double(x):
    return x * 2

Dorsy

but you end up with the same class instance for each application of the decorator. That is a problem, as each of the decorated functions is seing the same set of parameters. – Nazareth

Answer 6

I made a helper decorator for this purpose:

from functools import update_wrapper

class ClassWrapper:
    def __init__(self, cls):
        self.cls = cls
    
    def __call__(self, *args, **kwargs):
        class ClassWrapperInner:
            def __init__(self, cls, *args, **kwargs):
                # This combines previous information to get ready to recieve the actual function in the __call__ method.
                self._cls = cls
                self.args = args
                self.kwargs = kwargs
            
            def __call__(self, func, *args, **kw):
                # Basically "return self._cls(func, *self.args, **self.kwargs)", but with an adjustment to update the info of the new class & verify correct arguments
                assert len(args) == 0 and len(kw) == 0 and callable(func), f"{self._cls.__name__} got invalid arguments. Did you forget to parenthesize?"
                obj = self._cls(func, *self.args, **self.kwargs)
                update_wrapper(obj, func)
                return obj
            
        return ClassWrapperInner(self.cls, *args, **kwargs)

This weird code makes more sense in the context of how it will be executed:

double = ClassWrapper(Cache)(max_hits=100, timeout=50)(double)

ClassWrapper.__init__ stores the class it will be wrapping, (Cache).

ClassWrapper.__call__ passes on its arguments (max_hits=100, timeout=50) to ClassWrapperInner.__init__, which stores them for the next call.

ClassWrapper.__call__ combines all of the previous arguments and (func) together and gives them to an instance of your class, Cache, which it returns for use as the new double. It also updates your class's arguments, __name__ and __doc__ with the functools library. It's kind of like a way more complicated version of 2d list flattening where it's function arguments instead of lists.

With this class decorating it, your original function behaves as expected, except that you need to put parentheses around it in all cases.

@ClassWrapper
class Cache(object):
    def __init__(self, function, max_hits=10, timeout=5):
        self.function = function
        self.max_hits = max_hits
        self.timeout = timeout
        self.cache = {}

    def __call__(self, *args):
        ... # Here the code returning the correct thing.

@Cache()
def double(x):
    return x * 2

@Cache(max_hits=100, timeout=50)
def double(x):
    return x * 2

You could try to edit ClassWrapperInner.__call__ so that the parentheses are not required, but this approach is hacky and doesn't really make sense; it's like trying to add logic to each method of a class so that calling them without a self parameter works correctly.

EDIT: After writing this answer, I realized there was a much better way to make the decorator:

def class_wrapper(cls):
    def decorator1(*args, **kwargs):
        def decorator2(func):
            return cls(func, *args, **kwargs)
        return decorator2
    return decorator1

With functools functions for updating the name & things:

def class_wrapper(cls):
    def decorator1(*args, **kwargs):
        @wraps(cls)
        def decorator2(func):
            obj = cls(func, *args, **kwargs)
            update_wrapper(obj, func)
            return obj
        return decorator2
    return decorator1

Reconcilable

Answer 7

Define decorator that takes optional argument:

from functools import wraps, partial             
def _cache(func=None, *, instance=None):         
    if func is None:                             
        return partial(_cache, instance=instance)
    @wraps(func)                                 
    def wrapper(*ar, **kw):                      
        print(instance)                          
        return func(*ar, **kw)                   
    return wrapper         

And pass the instance object to decorator in __call__, or use other helper class that is instantiated on each __call__. This way you can use decorator without brackets, with params or even define a __getattr__ in proxy Cache class to apply some params.

class Cache:                                   
    def __call__(self, *ar, **kw):             
        return _cache(*ar, instance=self, **kw)
                                               
cache = Cache()                                
                                               
@cache                                         
def f(): pass                                  
f() # prints <__main__.Cache object at 0x7f5c1bde4880>

                                       

                  

Fahland

Answer 8

class myclass2:
 def __init__(self,arg):
  self.arg=arg
  print("call to init")
 def __call__(self,func):
  print("call to __call__ is made")
  self.function=func
  def myfunction(x,y,z):
   return x+y+z+self.function(x,y,z)
  self.newfunction=myfunction
  return self.newfunction
 @classmethod
 def prints(cls,arg):
  cls.prints_arg=arg
  print("call to prints is made")
  return cls(arg)


@myclass2.prints("x")
def myfunction1(x,y,z):
 return x+y+z
print(myfunction1(1,2,3))

remember it goes like this:
first call return object get second argument
usually if applicable it goes like argument,function,old function arguments

Hughhughes

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Phoenicia

Answer 9

You can also implement the class decorator using a metaclass. The __call__ method of the metaclass will wrap the original decorator when the decorator is used with keyword arguments.

class CacheMeta(type):
    def __call__(cls, *args, **kwargs):
        factory = super().__call__

        def wrap(function):
            return factory(function, **kwargs)

        return wrap if kwargs and not args else wrap(*args)


class Cache(metaclass=CacheMeta):
    def __init__(self, function, max_hits=10, timeout=5):
        self.function = function
        self.max_hits = max_hits
        self.timeout = timeout
        self.cache = {}

    def __call__(self, *args):
        # Here the code returning the correct thing.


@Cache
def double(x):
    return x * 2

@Cache(max_hits=100, timeout=50)
def double(x):
    return x * 2

Conscientious

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. – Phoenicia