Can a decorator of an instance method access the class?

Question

I have something roughly like the following. Basically I need to access the class of an instance method from a decorator used upon the instance method in its definition.

def decorator(view):
    # do something that requires view's class
    print view.im_class
    return view

class ModelA(object):
    @decorator
    def a_method(self):
        # do some stuff
        pass

The code as-is gives:

AttributeError: 'function' object has no attribute 'im_class'

I found similar question/answers - Python decorator makes function forget that it belongs to a class and Get class in Python decorator - but these rely upon a workaround that grabs the instance at run-time by snatching the first parameter. In my case, I will be calling the method based upon the information gleaned from its class, so I can't wait for a call to come in.

Friarbird

Answer 1

If you are using Python 2.6 or later you could use a class decorator, perhaps something like this (warning: untested code).

def class_decorator(cls):
   for name, method in cls.__dict__.iteritems():
        if hasattr(method, "use_class"):
            # do something with the method and class
            print name, cls
   return cls

def method_decorator(view):
    # mark the method as something that requires view's class
    view.use_class = True
    return view

@class_decorator
class ModelA(object):
    @method_decorator
    def a_method(self):
        # do some stuff
        pass

The method decorator marks the method as one that is of interest by adding a "use_class" attribute - functions and methods are also objects, so you can attach additional metadata to them.

After the class has been created the class decorator then goes through all the methods and does whatever is needed on the methods that have been marked.

If you want all the methods to be affected then you could leave out the method decorator and just use the class decorator.

Vertebral

Thanks I think this is the route with which to go. Just one extra line of code for any class I'd want to use this decorator. Maybe I could use a custom metaclass and perform this same check during new...? – Friarbird

Anyone trying to use this with staticmethod or classmethod will want to read this PEP: python.org/dev/peps/pep-0232 Not sure it's possible because you can't set an attribute on a class/static method and I think they gobble up any custom function attributes when they are applied to a function. – Friarbird

Just what I was looking for, for my DBM based ORM... Thanks, dude. – Parturition

You should use inspect.getmro(cls) to process all base classes in the class decorator to support inheritance. – Trigger

@Trigger do you mean instead of cls.__dict__? to first get the mro classes, then iterate over them and do cls.__dict__ for each? – Mcmorris

oh, actually it looks like inspect to the rescue https://mcmap.net/q/48826/-how-do-i-get-list-of-methods-in-a-python-class – Mcmorris

How can I make this method decorator accept values? I'm thinking @method_decorator('foobar') should result in view.use_class = 'foobar'. – Laundromat

PLEASE NOTE: in Python 3, if you are willing to put up with string parsing, the function passed in to a decorator will contain the __qualname__ dundermember, which will be <Class>.<function name> I presume that it will be a class PATH if you have embedded classes. If the function is global, you just get the function name, no module name. – Latty

In Python 3 the method object has the self attribute. Calling type on that will return the class in which the method is defined. – Coycoyle

Answer 2

Since python 3.6 you can use object.__set_name__ to accomplish this in a very simple way. The doc states that __set_name__ is "called at the time the owning class owner is created". Here is an example:

class class_decorator:
    def __init__(self, fn):
        self.fn = fn

    def __set_name__(self, owner, name):
        # do something with owner, i.e.
        print(f"decorating {self.fn} and using {owner}")
        self.fn.class_name = owner.__name__

        # then replace ourself with the original method
        setattr(owner, name, self.fn)

Notice that it gets called at class creation time:

>>> class A:
...     @class_decorator
...     def hello(self, x=42):
...         return x
...
decorating <function A.hello at 0x7f9bedf66bf8> and using <class '__main__.A'>
>>> A.hello
<function __main__.A.hello(self, x=42)>
>>> A.hello.class_name
'A'
>>> a = A()
>>> a.hello()
42

If you want to know more about how classes are created and in particular exactly when __set_name__ is called, you can refer to the documentation on "Creating the class object".

Borneol

How would that look like for using the decorator with parameters? E.g. @class_decorator('test', foo='bar') – Laundromat

@Laundromat You can approach it similar to normal decorators that take arguments. Just have def decorator(*args, **kwds): class Descriptor: ...; return Descriptor – Thine

Wow, thank you very much. Didn't know about __set_name__ although I've been using Python 3.6+ for a long time. – Devious

There is one drawback of this method: the static checker does not understand this at all. Mypy will think that hello is not a method, but instead is an object of type class_decorator. – Devious

@Devious If nothing else works, you can use an if TYPE_CHECKING to define class_decorator as a normal decorator returning the correct type. – Borneol

I think this is not reliable, because decorator might be stacked in combination with other decorators. So if it is wrapped by another decorator than it might not be called. – Exocrine

@Exocrine thank you so much for that comment. I was trying to debug why this was not working for me. Do you know what the solution would be to use this with the @classmethod decorator as well? – Neuroglia

@kawing-chiu, you can address this issue by adding a dummy __call__ method to the class_decorator, one what calls the single line pass removes this issue. – Rothenberg

This method is very slick. But does it qualify as a hijacking of the __set_name__ method? It's not "setting" a "name" so is a bit surprising to readers of the code. – Epilogue

Answer 3

As others have pointed out, the class hasn't been created at the time the decorator is called. However, it's possible to annotate the function object with the decorator parameters, then re-decorate the function in the metaclass's __new__ method. You'll need to access the function's __dict__ attribute directly, as at least for me, func.foo = 1 resulted in an AttributeError.

Pekin

setattr should be used instead of accessing __dict__ – Trigger

Answer 4

As Mark suggests:

1. Any decorator is called BEFORE class is built, so is unknown to the decorator.
2. We can tag these methods and make any necessary post-process later.
3. We have two options for post-processing: automatically at the end of the class definition or somewhere before the application will run. I prefer the 1st option using a base class, but you can follow the 2nd approach as well.

This code shows how this may works using automatic post-processing:

def expose(**kw):
    "Note that using **kw you can tag the function with any parameters"
    def wrap(func):
        name = func.func_name
        assert not name.startswith('_'), "Only public methods can be exposed"

        meta = func.__meta__ = kw
        meta['exposed'] = True
        return func

    return wrap

class Exposable(object):
    "Base class to expose instance methods"
    _exposable_ = None  # Not necessary, just for pylint

    class __metaclass__(type):
        def __new__(cls, name, bases, state):
            methods = state['_exposed_'] = dict()

            # inherit bases exposed methods
            for base in bases:
                methods.update(getattr(base, '_exposed_', {}))

            for name, member in state.items():
                meta = getattr(member, '__meta__', None)
                if meta is not None:
                    print "Found", name, meta
                    methods[name] = member
            return type.__new__(cls, name, bases, state)

class Foo(Exposable):
    @expose(any='parameter will go', inside='__meta__ func attribute')
    def foo(self):
        pass

class Bar(Exposable):
    @expose(hide=True, help='the great bar function')
    def bar(self):
        pass

class Buzz(Bar):
    @expose(hello=False, msg='overriding bar function')
    def bar(self):
        pass

class Fizz(Foo):
    @expose(msg='adding a bar function')
    def bar(self):
        pass

print('-' * 20)
print("showing exposed methods")
print("Foo: %s" % Foo._exposed_)
print("Bar: %s" % Bar._exposed_)
print("Buzz: %s" % Buzz._exposed_)
print("Fizz: %s" % Fizz._exposed_)

print('-' * 20)
print('examine bar functions')
print("Bar.bar: %s" % Bar.bar.__meta__)
print("Buzz.bar: %s" % Buzz.bar.__meta__)
print("Fizz.bar: %s" % Fizz.bar.__meta__)

The output yields:

Found foo {'inside': '__meta__ func attribute', 'any': 'parameter will go', 'exposed': True}
Found bar {'hide': True, 'help': 'the great bar function', 'exposed': True}
Found bar {'msg': 'overriding bar function', 'hello': False, 'exposed': True}
Found bar {'msg': 'adding a bar function', 'exposed': True}
--------------------
showing exposed methods
Foo: {'foo': <function foo at 0x7f7da3abb398>}
Bar: {'bar': <function bar at 0x7f7da3abb140>}
Buzz: {'bar': <function bar at 0x7f7da3abb0c8>}
Fizz: {'foo': <function foo at 0x7f7da3abb398>, 'bar': <function bar at 0x7f7da3abb488>}
--------------------
examine bar functions
Bar.bar: {'hide': True, 'help': 'the great bar function', 'exposed': True}
Buzz.bar: {'msg': 'overriding bar function', 'hello': False, 'exposed': True}
Fizz.bar: {'msg': 'adding a bar function', 'exposed': True}

Note that in this example:

1. We can annotate any function with any arbitrary parameters.
2. Each class has its own exposed methods.
3. We can inherit exposed methods as well.
4. methods can be overriding as exposing feature is updated.

Hope this helps

Trespass

Answer 5

The problem is that when the decorator is called the class doesn't exist yet. Try this:

def loud_decorator(func):
    print("Now decorating %s" % func)
    def decorated(*args, **kwargs):
        print("Now calling %s with %s,%s" % (func, args, kwargs))
        return func(*args, **kwargs)
    return decorated

class Foo(object):
    class __metaclass__(type):
        def __new__(cls, name, bases, dict_):
            print("Creating class %s%s with attributes %s" % (name, bases, dict_))
            return type.__new__(cls, name, bases, dict_)

    @loud_decorator
    def hello(self, msg):
        print("Hello %s" % msg)

Foo().hello()

This program will output:

Now decorating <function hello at 0xb74d35dc>
Creating class Foo(<type 'object'>,) with attributes {'__module__': '__main__', '__metaclass__': <class '__main__.__metaclass__'>, 'hello': <function decorated at 0xb74d356c>}
Now calling <function hello at 0xb74d35dc> with (<__main__.Foo object at 0xb74ea1ac>, 'World'),{}
Hello World

As you see, you are going to have to figure out a different way to do what you want.

Largehearted

when one defines a function the function doesn't exist yet, but one is able to recursively call the function from within itself. I guess this is a language feature specific to functions and not available to classes. – Friarbird

DGGenuine: The function is only called, and the function thus accesses itself, only after it was created completely. In this case, the class can not be complete when the decorator is called, since the class must wait for the decorator's result, which will be stored as one of the attributes of the class. – Ka

Answer 6

As Ants indicated, you can't get a reference to the class from within the class. However, if you're interested in distinguishing between different classes ( not manipulating the actual class type object), you can pass a string for each class. You can also pass whatever other parameters you like to the decorator using class-style decorators.

class Decorator(object):
    def __init__(self,decoratee_enclosing_class):
        self.decoratee_enclosing_class = decoratee_enclosing_class
    def __call__(self,original_func):
        def new_function(*args,**kwargs):
            print 'decorating function in ',self.decoratee_enclosing_class
            original_func(*args,**kwargs)
        return new_function


class Bar(object):
    @Decorator('Bar')
    def foo(self):
        print 'in foo'

class Baz(object):
    @Decorator('Baz')
    def foo(self):
        print 'in foo'

print 'before instantiating Bar()'
b = Bar()
print 'calling b.foo()'
b.foo()

Prints:

before instantiating Bar()
calling b.foo()
decorating function in  Bar
in foo

Also, see Bruce Eckel's page on decorators.

Budgie

Thanks for confirming my depressing conclusion that this isn't possible. I could also use a string that fully qualified the module/class ('module.Class'), store the string(s) until the classes have all fully loaded, then retrieve the classes myself with import. That seems like a woefully un-DRY way to accomplish my task. – Friarbird

You don't need to use a class for this sort of decorator: the idiomatic approach is to use one extra level of nested functions inside the decorator function. However, if you do go with classes, it might be nicer to not use capitalisation in the class name to make the decoration itself look "standard", i.e. @decorator('Bar') as opposed to @Decorator('Bar'). – Belgian

Answer 7

What flask-classy does is create a temporary cache that it stores on the method, then it uses something else (the fact that Flask will register the classes using a register class method) to actually wraps the method.

You can reuse this pattern, this time using a metaclass so that you can wrap the method at import time.

def route(rule, **options):
    """A decorator that is used to define custom routes for methods in
    FlaskView subclasses. The format is exactly the same as Flask's
    `@app.route` decorator.
    """

    def decorator(f):
        # Put the rule cache on the method itself instead of globally
        if not hasattr(f, '_rule_cache') or f._rule_cache is None:
            f._rule_cache = {f.__name__: [(rule, options)]}
        elif not f.__name__ in f._rule_cache:
            f._rule_cache[f.__name__] = [(rule, options)]
        else:
            f._rule_cache[f.__name__].append((rule, options))

        return f

    return decorator

On the actual class (you could do the same using a metaclass):

@classmethod
def register(cls, app, route_base=None, subdomain=None, route_prefix=None,
             trailing_slash=None):

    for name, value in members:
        proxy = cls.make_proxy_method(name)
        route_name = cls.build_route_name(name)
        try:
            if hasattr(value, "_rule_cache") and name in value._rule_cache:
                for idx, cached_rule in enumerate(value._rule_cache[name]):
                    # wrap the method here

Source: https://github.com/apiguy/flask-classy/blob/master/flask_classy.py

Supernatural

that's a useful pattern, but this doesn't address the problem of a method decorator being able to refer to the parent class of the method it's applied to – Mcmorris

I updated my answer to be more explicit how this can be useful to get access to the class at import time (i.e. using a metaclass + caching the decorator param on the method). – Supernatural

Answer 8

Here's a simple example:

def mod_bar(cls):
    # returns modified class

    def decorate(fcn):
        # returns decorated function

        def new_fcn(self):
            print self.start_str
            print fcn(self)
            print self.end_str

        return new_fcn

    cls.bar = decorate(cls.bar)
    return cls

@mod_bar
class Test(object):
    def __init__(self):
        self.start_str = "starting dec"
        self.end_str = "ending dec" 

    def bar(self):
        return "bar"

The output is:

>>> import Test
>>> a = Test()
>>> a.bar()
starting dec
bar
ending dec

Ptisan

Answer 9

As other answers have pointed out, decorator is an function-ish thing, you can not access the class which this method belongs to since the class has not been created yet. However, it's totally ok to use a decorator to "mark" the function and then use metaclass techniques to deal with the method later, because at the __new__ stage, the class has been created by its metaclass.

Here is a simple example:

We use @field to mark the method as a special field and deal with it in metaclass.

def field(fn):
    """Mark the method as an extra field"""
    fn.is_field = True
    return fn

class MetaEndpoint(type):
    def __new__(cls, name, bases, attrs):
        fields = {}
        for k, v in attrs.items():
            if inspect.isfunction(v) and getattr(k, "is_field", False):
                fields[k] = v
        for base in bases:
            if hasattr(base, "_fields"):
                fields.update(base._fields)
        attrs["_fields"] = fields

        return type.__new__(cls, name, bases, attrs)

class EndPoint(metaclass=MetaEndpoint):
    pass


# Usage

class MyEndPoint(EndPoint):
    @field
    def foo(self):
        return "bar"

e = MyEndPoint()
e._fields  # {"foo": ...}

Unship

You have a typo in this line: if inspect.isfunction(v) and getattr(k, "is_field", False): it should be getattr(v, "is_field", False) instead. – Coeducation

Answer 10

Function doesn't know whether it's a method at definition point, when the decorator code runs. Only when it's accessed via class/instance identifier it may know its class/instance. To overcome this limitation, you may decorate by descriptor object to delay actual decorating code until access/call time:

class decorated(object):
    def __init__(self, func, type_=None):
        self.func = func
        self.type = type_

    def __get__(self, obj, type_=None):
        func = self.func.__get__(obj, type_)
        print('accessed %s.%s' % (type_.__name__, func.__name__))
        return self.__class__(func, type_)

    def __call__(self, *args, **kwargs):
        name = '%s.%s' % (self.type.__name__, self.func.__name__)
        print('called %s with args=%s kwargs=%s' % (name, args, kwargs))
        return self.func(*args, **kwargs)

This allows you to decorate individual (static|class) methods:

class Foo(object):
    @decorated
    def foo(self, a, b):
        pass

    @decorated
    @staticmethod
    def bar(a, b):
        pass

    @decorated
    @classmethod
    def baz(cls, a, b):
        pass

class Bar(Foo):
    pass

Now you can use decorator code for introspection...

>>> Foo.foo
accessed Foo.foo
>>> Foo.bar
accessed Foo.bar
>>> Foo.baz
accessed Foo.baz
>>> Bar.foo
accessed Bar.foo
>>> Bar.bar
accessed Bar.bar
>>> Bar.baz
accessed Bar.baz

...and for changing function behavior:

>>> Foo().foo(1, 2)
accessed Foo.foo
called Foo.foo with args=(1, 2) kwargs={}
>>> Foo.bar(1, b='bcd')
accessed Foo.bar
called Foo.bar with args=(1,) kwargs={'b': 'bcd'}
>>> Bar.baz(a='abc', b='bcd')
accessed Bar.baz
called Bar.baz with args=() kwargs={'a': 'abc', 'b': 'bcd'}

Markley

Sadly, this approach is functionally equivalent to Will McCutchen's equally inapplicable answer. Both this and that answer obtain the desired class at method call time rather than method decoration time, as required by the original question. The only reasonable means of obtaining this class at a sufficiently early time is to introspect over all methods at class definition time (e.g., via a class decorator or metaclass). </sigh> – Diego

Answer 11

This is an old question but came across venusian. http://venusian.readthedocs.org/en/latest/

It seems to have the ability to decorate methods and give you access to both the class and the method while doing so. Note tht calling setattr(ob, wrapped.__name__, decorated) is not the typical way of using venusian and somewhat defeats the purpose.

Either way... the example below is complete and should run.

import sys
from functools import wraps
import venusian

def logged(wrapped):
    def callback(scanner, name, ob):
        @wraps(wrapped)
        def decorated(self, *args, **kwargs):
            print 'you called method', wrapped.__name__, 'on class', ob.__name__
            return wrapped(self, *args, **kwargs)
        print 'decorating', '%s.%s' % (ob.__name__, wrapped.__name__)
        setattr(ob, wrapped.__name__, decorated)
    venusian.attach(wrapped, callback)
    return wrapped

class Foo(object):
    @logged
    def bar(self):
        print 'bar'

scanner = venusian.Scanner()
scanner.scan(sys.modules[__name__])

if __name__ == '__main__':
    t = Foo()
    t.bar()

Welladvised

Answer 12

You will have access to the class of the object on which the method is being called in the decorated method that your decorator should return. Like so:

def decorator(method):
    # do something that requires view's class
    def decorated(self, *args, **kwargs):
        print 'My class is %s' % self.__class__
        method(self, *args, **kwargs)
    return decorated

Using your ModelA class, here is what this does:

>>> obj = ModelA()
>>> obj.a_method()
My class is <class '__main__.ModelA'>

Uphold

Thanks but this is exactly the solution I referenced in my question that doesn't work for me. I am trying to implement an observer pattern using decorators and I will never be able to call the method in the correct context from my observation dispatcher if I don't have the class at some point while adding the method to the observation dispatcher. Getting the class upon method call doesn't help me correctly call the method in the first place. – Friarbird

Whoa, sorry for my laziness in not reading your entire question. – Uphold

Answer 13

I just want to add my example since it has all the things I could think of for accessing the class from the decorated method. It uses a descriptor as @tyrion suggests. The decorator can take arguments and passes them to the descriptor. It can deal with both a method in a class or a function without a class.

import datetime as dt
import functools

def dec(arg1):
    class Timed(object):
        local_arg = arg1
        def __init__(self, f):
            functools.update_wrapper(self, f)
            self.func = f

        def __set_name__(self, owner, name):
            # doing something fancy with owner and name
            print('owner type', owner.my_type())
            print('my arg', self.local_arg)

        def __call__(self, *args, **kwargs):
            start = dt.datetime.now()
            ret = self.func(*args, **kwargs)
            time = dt.datetime.now() - start
            ret["time"] = time
            return ret
        
        def __get__(self, instance, owner):
            from functools import partial
            return partial(self.__call__, instance)
    return Timed

class Test(object):
    def __init__(self):
        super(Test, self).__init__()

    @classmethod
    def my_type(cls):
        return 'owner'

    @dec(arg1='a')
    def decorated(self, *args, **kwargs):
        print(self)
        print(args)
        print(kwargs)
        return dict()

    def call_deco(self):
        self.decorated("Hello", world="World")

@dec(arg1='a function')
def another(*args, **kwargs):
    print(args)
    print(kwargs)
    return dict()

if __name__ == "__main__":
    t = Test()
    ret = t.call_deco()
    another('Ni hao', world="shi jie")
    

Editorialize

Answer 14

@asterio gonzalez

I prefer your method, however it has to be changed a little for Python 3 to comply with the new metaclass processing (also, some print statements were missing parentheses):

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Created on Mon Aug  9 15:27:30 2021

@author: yves
"""

def expose(**kw):
    "Note that using **kw you can tag the function with any parameters"
    def wrap(func):
        name = func.__name__
        assert not name.startswith('_'), "Only public methods can be exposed"

        meta = func.__meta__ = kw
        meta['exposed'] = None
        return func

    return wrap

class ExposableMetaclass(type):
    def __new__(cls, name, bases, state):
        methods = state['_exposed_'] = dict()

        # inherit bases exposed methods
        for base in bases:
            methods.update(getattr(base, '_exposed_', {}))

        for name, member in state.items():
            meta = getattr(member, '__meta__', None)
            if meta is not None:
                print("Found", name, meta)
                methods[name] = member
        return type.__new__(cls, name, bases, state)

class Exposable(metaclass=ExposableMetaclass):
    "Base class to expose instance methods"
    _exposable_ = None  # Not necessary, just for pylint

class Foo(Exposable):
    @expose(any='parameter will go', inside='__meta__ func attribute')
    def foo(self):
        pass

class Bar(Exposable):
    @expose(hide=True, help='the great bar function')
    def bar(self):
        pass

class Buzz(Bar):
    @expose(hello=False, msg='overriding bar function')
    def bar(self):
        pass

class Fizz(Foo):
    @expose(msg='adding a bar function')
    def bar(self):
        pass

print('-' * 20)
print("showing exposed methods")
print("Foo: %s" % Foo._exposed_)
print("Bar: %s" % Bar._exposed_)
print("Buzz: %s" % Buzz._exposed_)
print("Fizz: %s" % Fizz._exposed_)

Met my needs!

Pronounce