Passing 'self' parameter during methods decorating in Python [duplicate]

Question

I want to create decorator that shows which parameters were passed to function and methods. I have already written the code for functions, but methods are giving me a headaches.

This is function decorator that works as intended:

from functools import update_wrapper


class _PrintingArguments:
    def __init__(self, function, default_comment, comment_variable):
        self.function = function
        self.comment_variable = comment_variable
        self.default_comment = default_comment
        update_wrapper(wrapped=function, wrapper=self)

    def __call__(self, *args, **kwargs):
        comment = kwargs.pop(self.comment_variable, self.default_comment)
        params_str = [repr(arg) for arg in args] + ["{}={}".format(k, repr(v)) for k, v in kwargs.items()]
        function_call_log = "{}({})".format(self.function.__name__, ", ".join(params_str))
        print("Function execution - '{}'\n\t{}".format(comment, function_call_log))
        function_return = self.function(*args, **kwargs)
        print("\tFunction executed\n")
        return function_return


def function_log(_function=None, default_comment="No comment.", comment_variable="comment"):
    if _function is None:
        def decorator(func):
            return _PrintingArguments(function=func, default_comment=default_comment, comment_variable=comment_variable)
        return decorator
    else:
        return _PrintingArguments(function=_function, default_comment=default_comment, comment_variable=comment_variable)

# example use:
@function_log
def a(*args, **kwargs):
    pass


@function_log(default_comment="Hello World!", comment_variable="comment2")
def b(*args, **kwargs):
    pass


a(0, x=1, y=2)
a(0, x=1, y=2, comment="Custom comment!")

b("a", "b", "c", asd="something")
b("a", "b", "c", asd="something", comment2="Custom comment for b!")

Output of the code execution:

Function execution - 'No comment.'
    a(0, y=2, x=1)
    Function executed

Function execution - 'Custom comment!'
    a(0, y=2, x=1)
    Function executed

Function execution - 'Hello World!'
    b('a', 'b', 'c', asd='something')
    Function executed

Function execution - 'Custom comment for b!'
    b('a', 'b', 'c', asd='something')
    Function executed

I have tried the exactly same decorator for methods:

class A:
    def __init__(self):
        pass

    @function_log
    def method1(self, *args, **kwargs):
        print("\tself = {}".format(self))

    @function_log(default_comment="Something", comment_variable="comment2")
    def method2(self, *args, **kwargs):
        print("\tself = {}".format(self))

a_obj = A()

a_obj.method1(0, 1, p1="abc", p2="xyz")
a_obj.method1(0, 1, p1="abc", p2="xyz", comment="My comment")

a_obj.method2("a", "b", p1="abc", p2="xyz")
a_obj.method2("a", "b", p1="abc", p2="xyz", comment="My comment 2")

The output is:

Function execution - 'No comment.'
    method1(0, 1, p2='xyz', p1='abc')
    self = 0
    Function executed

Function execution - 'My comment'
    method1(0, 1, p2='xyz', p1='abc')
    self = 0
    Function executed

Function execution - 'Something'
    method2('a', 'b', p2='xyz', p1='abc')
    self = a
    Function executed

Function execution - 'Something'
    method2('a', 'b', comment='My comment 2', p2='xyz', p1='abc')
    self = a
    Function executed

Parameter 'self' is not passed by my decorator to the method.
I want to write second decorator 'method_log' that would work pretty similar as 'function_log'. For code:

class A:
    def __init__(self):
        pass

    @method_log
    def method1(self, *args, **kwargs):
        print("\tself = {}".format(self))

    @fmethod_log(default_comment="Something", comment_variable="comment2")
    def method2(self, *args, **kwargs):
        print("\tself = {}".format(self))

a_obj = A()

a_obj.method1(0, 1, p1="abc", p2="xyz")
a_obj.method1(0, 1, p1="abc", p2="xyz", comment="My comment")

a_obj.method2("a", "b", p1="abc", p2="xyz")
a_obj.method2("a", "b", p1="abc", p2="xyz", comment="My comment 2")

I want the output:

Method execution - 'No comment.'
    method1(<__main__.A instance at ...>, 0, 1, p2='xyz', p1='abc')
    self = <__main__.A instance at ...> #
    Function executed

Method execution - 'My comment'
    method1(<__main__.A instance at ...>, 0, 1, p2='xyz', p1='abc')
    self = <__main__.A instance at ...>
    Function executed

Method execution - 'Something'
    method2(<__main__.A instance at ...>, 'a', 'b', p2='xyz', p1='abc')
    self = <__main__.A instance at ...>
    Function executed

Method execution - 'Something'
    method2(<__main__.A instance at ...>, 'a', 'b', comment='My comment 2', p2='xyz', p1='abc')
    self = <__main__.A instance at ...>
    Function executed

Defection

https://mcmap.net/q/206543/-access-self-from-decorator – Screwworm

@Screwworm Despite similar titles, the question is actually substantially different. – Payday

@JBChouinard Well, that's what I get for not reading the it thoroughly lol – Screwworm

Answer 1

It's not working with you current design because of how classes work in Python.

When a class is instantiated, the functions on it get bound to the instance - they become bound methods, so that self is automatically passed.

You can see it happen:

class A:
    def method1(self):
        pass

>>> A.method1
<function A.method1 at 0x7f303298ef28>
>>> a_instance = A()
>>> a_instance.method1
<bound method A.method1 of <__main__.A object at 0x7f303a36c518>>

When A is instantiated, method1 is magically transformed from a function into a bound method.

Your decorator replaces method1 - instead of a real function, it is now an instance of _PrintingArguments. The magic that turns functions into bound methods is not applied to random objects, even if they define __call__ so that they behave like a function. (But that magic can be applied, if your class implements the Descriptor protocol, see ShadowRanger's answer!).

class Decorator:
    def __init__(self, func):
        self.func = func

    def __call__(self, *args, **kwargs):
        return self.func(*args, **kwargs)


class A:
    @Decorator
    def method1(self):
        pass

>>> A.method1
<__main__.Decorator object at 0x7f303a36cbe0>
>>> a_instance = A()
>>> a_instance.method1
<__main__.Decorator object at 0x7f303a36cbe0>

There is no magic. method1 on the instance of A is not a bound method, it's just a random object with a __call__ method, which will not have self passed automatically.

If you want to decorate methods you have to replace the decorated function with another real function, an arbitrary object with __call__ will not do.

You could adapt your current code to return a real function:

import functools

class _PrintingArguments:
    def __init__(self, default_comment, comment_variable):
        self.comment_variable = comment_variable
        self.default_comment = default_comment

    def __call__(self, function):
        @functools.wraps(function)
        def decorated(*args, **kwargs):
            comment = kwargs.pop(self.comment_variable, self.default_comment)
            params_str = [repr(arg) for arg in args] + ["{}={}".format(k, repr(v)) for k, v in kwargs.items()]
            function_call_log = "{}({})".format(function.__name__, ", ".join(params_str))
            print("Function execution - '{}'\n\t{}".format(comment, function_call_log))
            function_return = function(*args, **kwargs)
            print("\tFunction executed\n")
            return function_return
        return decorated

def function_log(_function=None, default_comment="No comment.", comment_variable="comment"):
    decorator = _PrintingArguments(
        default_comment=default_comment,
        comment_variable=comment_variable,
    )
    if _function is None:
        return decorator
    else:
        return decorator(_function)

Payday

FYI, that magic isn't automatically applied to callable classes, but you can turn it on manually by implementing the descriptor protocol (def-ed functions automatically implement the descriptor protocol, but any class can do it). Sadly, the code to do so differs slightly between Py2 and Py3, but it's totally doable, with no changes to existing code (aside from explicitly inheriting from object on Py2), just new code. – Hanukkah

@Hanukkah I did not know that, very cool. – Payday

@Hanukkah Descriptors are a much better solution to some stuff I've been doing with metaclasses. – Payday

Yeah, I've generally found that with a few rare exceptions for extremely complex cases (e.g. enum.EnumMeta and abc.ABCMeta), as soon as any given feature of metaclasses is used semi-frequently, a non-metaclass based solution is provided in the language. Storing class attributes in declaration order used to require a metaclass, now they're ordered for free; class decorators replaced them for post-hoc class modification; __class_getitem__ replaced them for type generics; etc. The descriptor protocol is unusual only in that it released alongside metaclasses; it's not a later addition. – Hanukkah

I suspect a large part of this tendency to provide a more targeted solution to things metaclasses could already do was because: 1) The metaclass-based solution required either a (relatively simple, but still quite verbose) custom metaclass for every implementing class, or a ridiculously complex metaclass if you wanted to reuse the metaclass for many implementing classes with subtly different behaviors, 2) the metaclass was much slower, and most importantly, 3) more metaclass use dramatically increases the risk of a metaclass conflict occurring due to multiple inheritance. – Hanukkah

Thank you guys, I better understand what is implicitly happening behind methods and why callable object cannot replace them. – Defection

Answer 2

If you want _PrintingArguments to bind the same way as a plain function, this is actually possible, you just need to implement the descriptor protocol yourself to match the way built-in functions behave. Conveniently, Python provides types.MethodType, which can be used to create a bound method from any callable, given an instance to bind to, so we use that to implement our descriptor's __get__:

import types

class _PrintingArguments:
    # __init__ and __call__ unchanged

    def __get__(self, instance, owner):
        if instance is None:
            return self  # Accessed from class, return unchanged
        return types.MethodType(self, instance)  # Accessed from instance, bind to instance

This works as you expect on Python 3 (Try it online!). On Python 2 it's even simpler (because unbound methods exist, so the call to types.MethodType can be made unconditionally):

import types

class _PrintingArguments(object):  # Explicit inheritance from object needed for new-style class on Py2
    # __init__ and __call__ unchanged

    def __get__(self, instance, owner):
        return types.MethodType(self, instance, owner)  # Also pass owner

Try it online!

For slightly better performance (on Python 2 only), you could instead do:

class _PrintingArguments(object):  # Explicit inheritance from object needed for new-style class on Py2
    # __init__ and __call__ unchanged

# Defined outside class, immediately after dedent
_PrintingArguments.__get__ = types.MethodType(types.MethodType, None, _PrintingArguments)

which moves the implementation of __get__ to the C layer by making an unbound method out of types.MethodType itself, removing byte code interpreter overhead from each call.

Hanukkah

Side-note: To avoid needing to explicitly inherit from object for each class to make them new-style, you can add __metaclass__ = type to the top of each source file you want to be uniformly new-style; every class defined in that source file will be new-style automatically without explicit inheritance from object. – Hanukkah