Programmatically generate methods for a class

Question

I have about 20 methods to redirect to a wrapper method that takes the original method, and the rest of the arguments:

class my_socket(parent):

    def _in(self, method, *args, **kwargs):
        # do funky stuff

    def recv(self, *args, **kwargs):
        return self._in(super().recv, *args, **kwargs)

    def recv_into(self, *args, **kwargs):
        return self._in(super().recv_into, *args, **kwargs)

    # and so on...

How can I add more of these methods programmatically? This is about as far as I get before everything starts to look wrong:

for method in 'recv', 'recvfrom', 'recvfrom_into', 'recv_into', ...:
    setattr(my_socket, method, ???)

Can I do this by assigning within the class definition, or something else that feels more natural?

class my_socket(parent):

    def makes_recv_methods(name):
        # wraps call to name

    def recv_meh = makes_recv_methods('recv_meh')

I'd prefer to use __get__ and friends when possible over magic functions from types.

Skipbomb

Can't you refactor your code to do the funky stuff in a @decorator - or is there something I don't get here? – Setser

@Kimvais: Yeah but then how will I bind decorated function with a name back to the class? I'll end up with @decorator('recv_into'): def recv_into(self): pass, right? – Skipbomb

What's the point of super when there is no superclass? – Lavernelaverock

Answer 1

I'd do it by running some code to generate the methods from a list after the class is defined - you could put this into a decorator.

import functools

def wrap_method(cls, name):
    # This unbound method will be pulled from the superclass.
    wrapped = getattr(cls, name)
    @functools.wraps(wrapped)
    def wrapper(self, *args, **kwargs):
        return self._in(wrapped.__get__(self, cls), *args, **kwargs)
    return wrapper

def wrap_methods(cls):
    for name in cls.WRAP_ATTRS:
        setattr(cls, name, wrap_method(cls, name))
    return cls

@wrap_methods
class my_socket(parent_class):
    WRAP_ATTRS = ['recv', 'recvfrom'] # ... + more method names

    def _in(self, method, *args, **kwargs):
        # do funky stuff

Actinopod

I think Matt wants to call the method in _in, maybe after modifying the arguments. You could instead pass the bound method as the first argument: self._in(wrapped.__get__(self, cls), *args, **kwargs). – Scrubland

Ah, yes - changing to incorporate that. – Actinopod

After reading that nonsense I've decided against it. There's so much indirection it makes me nauseous. I hope we eventually get anonymous functions. – Skipbomb

Python has anonymous functions. There's nothing stopping wrapper (the only closure generated in this code) from being a lambda, other than good taste. ;) – Actinopod

Answer 2

wilberforce proposal works, but there is a simpler way using OOP only:

def wrap_method(wrapped):
    @functools.wraps(wrapped)
    def wrapper(self, *args, **kwargs):
        return self._in(wrapped.__get__(self, cls), *args, **kwargs)
    return wrapper

class Parent:

    def _in(self, method, *args, **kwargs):
        return method(*args, **kwargs)


    @wrap_method
    def recv(self, *args, **kwargs):
        return # whatever

    @wrap_method
    def recv_into(self, *args, **kwargs):
        return # whatever

class MySocket(Parent):

    def _in(self, method, *args, **kwargs):
        # do funky stuff

Feer

Unfortunately I cannot be making changes to the parent class. – Skipbomb

well, no harm I providing an alternative to meta-programming. I voted up wilberforce's answer, but I tend to agree with @Matt. The op is trying to achieve something dangerous. He should try to design his code so that he can get away without tricks like this. – Feer

Answer 3

I'd like to expand on the accepted answer. I wanted to potentially have a very long list of decorator methods applied to a very long list of methods.

import functools


def wrap_method(cls, name, wrapper_method_name):
    # This unbound method will be pulled from the superclass.
    wrapped = getattr(cls, name, wrapper_method_name)

    @functools.wraps(wrapped)
    def wrapper(self, *args, **kwargs):
        wrapper_method = getattr(self, wrapper_method_name)
        return wrapper_method(wrapped.__get__(self, cls), *args, **kwargs)

    return wrapper


def wrap_methods(cls):
    for wrapper_method_name in cls.WRAPPER_METHOD_NAMES:
        for name in cls.WRAPPED_METHODS:
            setattr(cls, name, wrap_method(cls, name, wrapper_method_name))
    return cls

And here is the class that wraps the original

@wrap_methods
class WrappedConnection(BaseConnection):
    """
    This class adds some quality-of-life improvements to the BaseConnection class.
    -WRAPPED_METHODS are wrapped by WRAPPER_METHOD_NAMES
    -wrappers can be toggled on and off.

    example:
    connection = WrappedConnection(show_messages=True, log_errors=False, keep_authenticated=False)

    default:
    connection = WrappedConnection(show_messages=False, log_errors=True, keep_authenticated=True)
    """
    WRAPPER_METHOD_NAMES = ['log_errors', 'keep_authenticated', 'show_messages']
    WRAPPED_METHODS = ['a_method', 'b_method', 'c_method', 'd_method']
    MESSAGE_OVERRIDE_MAP = {"a_method": "a_method_message_override_attribute", 
                            "b_method": "b_method_message_override_attribute"}

    def keep_authenticated(self, method, *args, **kwargs):
        """
        If the session has expired, the session is re-authenticated. The incident is logged by the default logger.
        This option can be turned off by setting keep_authenticated during initialization of a WrappedConnection object.
        - connection = WrappedConnection(keep_authenticated=False) # why would you ever do this


        :param method: (method) method to be wrapped
        :param args: (args) passed args
        :param kwargs: (kwargs) passed kwargs
        :return: (method) method wrapped by @keep_authenticated
        """
        response, expired_session = method(*args, **kwargs), None
        if response["errors"] and self._keep_authenticated:
            expired_session = list(filter(lambda x: 'expired session' in x, response["errors"]))
        if expired_session:
            self.__init__()
            logging.info('Session has been re-authenticated.')
            response = method(*args, **kwargs)
        return response

    def log_errors(self, method, *args, **kwargs):
        """
        If there is an error the incident is logged. This option can be turned off by setting log_errors
        during initialization of a WrappedConnection object.
        - connection = WrappedConnection(log_errors=False)

        :param method: (method) method to be wrapped
        :param args: (args) passed args
        :param kwargs: (kwargs) passed kwargs
        :return: (method) method wrapped by @log_errors
        """
        response = method(*args, **kwargs)
        if response["errors"] and self._log_errors:
            errors = response["errors"]
            logging.error(errors)
        return response

    def show_messages(self, method, *args, **kwargs):
        """
        Shows the xml that is sent during the request. This option can be turned on by setting show_messages during
        initialization of a WrappedConnection object.
        - connection = WrappedConnection(show_messages=True)

        :param method: (method) method to be wrapped
        :param args: (args) passed args
        :param kwargs: (kwargs) passed kwargs
        :return: (method) method wrapped by @show_messages
        """
        response = method(*args, **kwargs)
        if self._show_messages:
            message_override_attr = WrappedConnection.MESSAGE_OVERRIDE_MAP.get(method.__name__)
            if message_override_attr:
                message_override = getattr(self, message_override_attr)
                print(BeautifulSoup(message_override, "xml").prettify())
            else:
                self._show_message(method.__name__, *args, **kwargs)
        return response

    def __init__(self, *args, keep_authenticated=True, log_errors=True, show_messages=False, **kwargs):
        super(WrappedConnection, self).__init__(*args, **kwargs)
        self._keep_authenticated = keep_authenticated
        self._log_errors = log_errors
        self._show_messages = show_messages

Heliotaxis

Answer 4

You could use cog.

class MySocket(Parent):
"""[[[cog
import cog
l = ['in','out']
for item in l:
    cog.outl("def _{0}(self, method, *args, **kwargs):".format(item))

]]]"""
#[[[end]]]

This has the added advantages of easily being updated, not touching your code outside of the end comment, and you can twiddle the generated code if necessary.

I've successfully used cog for generating boilerplate on another project, mixed in with the non generated code. It started out reading an input file of instructions into a dictionary. Then for each section of boilerplate it used that piece of the dictionary to know what to write.

I edit the instruction file in one spot, instead of twenty different places in the boilerplate.

Hartsell

IMHO, code generation is never needed in python, since you can always find a way to refactor boilerplate – Feer

@Simon The primary language I used cog with wasn't python. :) And depending on what he wants to do, code generation makes perfect sense. That's what a compiler or interpreter does, takes your instructions and outputs code. – Hartsell