Redefinition of class method in python

Question

Context

I'm trying to have some "plugins" (I'm not sure this is the correct definition for this) to my code. By "plugin", I mean a module which defines a model (this is a scientific code) in such a way that its existence is enough to use it anywhere else in the code.

Of course these plugins must follow a template which uses some modules/function/classes defined in my code. Here is a small snippet for the relevant part of my code:

# [In the code]
class AllModels():
    def __init__(self):
        self.count = 0

    def register(self, name, model):
        """
        Adds a model to the code
        """
        setattr(self, name, model)
        self.count += 1
        return

class Model():
    def __init__(self, **kwargs):
        """
        Some constants that defines a model
        """
        self.a = kwargs.get("a", None)
        self.b = kwargs.get("b", None)
        # and so on...

    def function1(self, *args, **kwargs):
        """
        A function that all models will have, but which needs:
            - to have a default behavior (when the instance is created)
            - to be redefinable by the "plugin" (ie. the model)
        """
        # default code for the default behavior
        return

instance = AllModels()

and here is the relevant part of the "plugin":

# [in the plugin file]
from code import Model, instance
newmodel = Model(a="a name", b="some other stuff")

def function1(*args, **kwargs):
    """
    Work to do by this model
    """
    # some specific model-dependent work
    return

instance.register(newmodel)

Additional information and requirements

• function1 has exactly the same signature for any model plugin, but is usually doing a different job for each.

• I'd like a default behavior for the function1 so that if it is not defined by the plugin, I'll still be able to do something (try different possibilities, and/or raise a warning/error).

• In the plugin, function1 may use some other functions that are only defined in this plugin. I'm stating this because the code is running with the multiprocessing module, and I need the instance instance of AllModels to be able to call function1 in child processes. instance is defined in the parent process, as well as the model plugins, but will be used in different child processes (no modification is done to it though).

• it would be awesome that function1, when "redefined" by the plugin, be able to access the attributes of the Model instance (ie. self).

Problem

I've read many different sources of python documentation and some SO question. I only see two/three possible solutions to this problem:

1. not declaring function1 method in Model class, but just set it as an attribute when the plugin creates a new instance of it.

   [in the plugin file]

   def function1(*args, **kwargs): # .... return newmodel.function1 = function1

and then call it whenever needed. In that case the attribute function1 in the object Model would be initiate to None probably. One caveat of that is that there is no "default behaviour" for function1 (it has to be dealt in the code, eg. testing if instance.function1 is None: ...), and an even bigger one is that I can't access self this way...

2. using somehow the python decorators. I've never used this, and the documentation I've read is not that simple (I mean not straight forward due to the huge number of possibilities on its usage). But it seems to be a good solution. However I'm worried about its performance impact (I've read that it could slow down the execution of the decorated function/method). If this solution is the best option, then I'd like to know how to use it (a quick snippet maybe), and if it is possible to use attributes of the class Model:

   [in the plugin file]

   @mydecorator def function1(self, *args, **kwargs): """ I'm not sure I can use self, but it would be great since some attributes of self are used for some other function similar to function1... """ # some stuff using self, eg.: x = self.var **2 + 3.4 # where self.var has been defined before, eg.: newmodel.var = 100.

3. using the module types and its MethodType... I'm not sure that is relevant in my case... but I may be wrong.

Brutish

It's better to keep the models in AllModels in a dict attribute instead of the class namespace. Otherwise you get crazy bugs when someone calls a model 'count' for example – Stefa

If this is python2, you need to make your classes new-style (inherit from object) if they don't inherit from anything else (like Model) – Berneta

@Berneta is it recommended to not use old-style class? Could you tell me why? – Brutish

@mhavel, new-style classes are bonafide user-defined types, but old-style classes are not; all instances of old-style classes are of the same type, instance. This means there are all kinds of Good Things that new-style classes can do that old-style classes can't. Plus old-style classes are being phased out. Basically, they're just yucky. – Godden

@Godden thanks for the explanation! I'll use new-style from now on. – Brutish

Answer 1

Well, unless I'm mistaken, you want to subclass Model. This is sort of like creating an instance of Model and replacing its function1 attribute with a function defined in the plugin module (i.e. your option 1); but it's much cleaner, and takes care of all the details for you:

# [in the plugin file]
from code import Model, instance

class MyModel(Model):
    def function1(*args, **kwargs):
        """
        Work to do by this model
        """
        # some specific model-dependent work
        return

newmodel = MyModel(a="a name", b="some other stuff")
instance.register(newmodel)

This way, all the other methods (functions "attached" to a Model instance) are inherited from Model; they will behave in just the same way, but function1 will be overridden, and will follow your customized function1 definition.

Godden

Answer 2

Could you write a dummy function1() function in the Model class and raise a NotImplementedError? That way, if anyone tries to inherit from Model without implementing function1(), they'll get an exception when they try to run the code. If you're running the code for them, you can catch that error and return a helpful error message to the user.

For example:

class Model:
    #Your code

    def function1():
        raise NotImplementedError("You need to implement function1 
            when you inherit from Model")

Then, you can do the following when you run the code:

try:
    modelObj.function1()
except NotImplementedError as e:
    #Perform error handling here

EDIT: The official Python documentation for NotImplementedError states: "In user defined base classes, abstract methods should raise this exception when they require derived classes to override the method." That does seem to fit the requirements here.

Aleksandrovsk

Thanks for your contribution. I'm not sure yet what would be the default behavior for function1, but raising an error is indeed a possibility. – Brutish

Answer 3

What you are trying to do canbe done in pretty straightforward ways - just using Objected Oriented techniques and taking advantage that in Python functions are also normal objects -

One simple thing to do is just having your "model" class to accept the "function1" as a parameter, and store it as an object member.

Some code like this, with minimal changes to your code - though much more interesting things are certainly possible:

# [In the code]

class AllModels():
    def __init__(self):
        """
        Init.
        """
        self.count = 0

    def register(self, name, **kwargs):
        """
        Adds a model to the code
        """
        model = Model(**kwargs)
        setattr(self, name, model)
        self.count += 1
        return

class Model():
    def __init__(self, **kwargs):
        """
        Some constants that defines a model
        """
        self.a = kwargs.get("a", None)
        self.b = kwargs.get("b", None)
        if "function1" in kwargs:
            self.real_function1 = kwargs["function1"]
            self.function1.__doc__ = kwargs["function1"].__doc__
        # and so on...

    def function1(self, *args, **kwargs):
        """
        A function that all models will have, but which needs:
            - to have a default behavior (when the instance is created)
            - to be redefinable by the "plugin" (ie. the model)
        """
        if self.real_function1:
            return self.real_function1(self, *args, **kwargs)
        # default code for the default behavior
        return

instance = AllModels()

and

# [in the plugin file]
from code import Model, instance
newmodel = Model(a="a name", b="some other stuff")

def function1(self, *args, **kwargs):
    """
    Work to do by this model
    """
    # some specific model-dependent work
    return


instance.register("name", function1 = function1, a="a name", b="some other stuff")

Innis

Interesting solution. But then, would it work with the class attributes (ie. accessing self)? From your snippet it seems to be the case, which would be nice. Thanks for your time. – Brutish

Yes, because the function would be manually called by the stub method in the Model class - this call wxplcily adds "self' in the parameters, in the suggestion above. – Innis