Description

I wonder if the code I am showing can be considered as an example of policy-based design in Python. Also, I would like to know if have you seen python modules using something like this example so I can learn from them?

I wrote more details and examples about this approach in a post.

Recently I needed something like policy-based design for a python module I was working on.

I found a similar question in this forum, but it was closed, and I was not able to add a comment.

Let me give a summary of what I expect from this approach in Python.

Module classes are divided between policy and host classes.
Policy classes implement modifications to the behavior or interface of the host classes by inheritance.
Users instantiate a new class from a host class by providing a set of policy classes.

Here it is my toy example:

class PrintOutput:
    """Implement print output policy."""
    def _output(self, message):
        print(message)

class SaveOutput:
    """Implement save output policy."""
    def set_filename(self, filename):
        self.filename = filename
    def _output(self, message):
        with open(self.filename, 'w') as file:
            file.write(message)

def HelloWorld(Output=PrintOutput):
    """Creates a host class."""
    class _(Output):
        """Implements class affected by the policy."""
        def run(self):
            """Print message."""
            self._output('Hello world!')
    return _

PrintHelloWorld = HelloWorld()
hw = PrintHelloWorld()
hw.run() # print "Hello World!"

SaveHelloWorld = HelloWorld(
    Output=SaveOutput
)
hw = SaveHelloWorld()
hw.set_filename('output.txt')
hw.run() # save "Hello World!" in output.txt

In the example, I expect that all the classes are already defined in a module. The user only needs to instantiate the HelloWorld class with the output policy that fits his or her requirements.

Design ingredients

The basic design ingredients are

Multiple inheritances: that exists in both C++ and Python.
Postpone inheritance: postponing the definition of the inheritance between host and policy classes until user instantiation of the host class

// in C++ using templates
template<typename Policy1, typename Policy2, ...>
class HostClass: public Policy1, public Policy2, ... {
    // Implement host class.
};

# in Python with a factory with arguments
def HostClass(Policy1, Policy2=Default2, ...):
    class _(Policy1, Policy2, ...):
        """Implements class affected by the policies."""
    return _

Class instantiation

// in C++
typedef HostClass<Policy1Class, Policy2Class, ...> NewClass;

# in Python
NewClass = HostClass(Policy1Class, Policy2Class, ...)

Using mixins as alternative

I just learned from one of the comments, that it is possible to use Python mixins as an alternative approach to creating new classes. Under this approach module's code is divided between based and mixin classes. You can then create new classes from the base classes, and using mixin classes to implement modifications to the behavior or interface provided by the base classes.

Following this answer, I was able to write my first example using mixins.

class PrintOutput:
    """Implement print output mixin."""
    def _output(self, message):
        print(message)

class SaveOutput:
    """Implement save output mixin."""
    def set_filename(self, filename):
        self.filename = filename
    def _output(self, message):
        with open(self.filename, 'w') as file:
            file.write(message)

class HelloWorld:
    """Creates a host class."""
    def run(self):
        """Print message."""
        self._output('Hello world!')

class PrintHelloWorld(PrintOutput, HelloWorld):
    pass

hw = PrintHelloWorld()
hw.run() # print "Hello World!"

class SaveHelloWorld(SaveOutput, HelloWorld):
    pass

hw = SaveHelloWorld()
hw.set_filename('output.txt')
hw.run() # save "Hello World!" in output.txt

The difference between approaches

The main difference between mixin and my previous example is the inheritance hierarchy between classes. Neither mixin nor based classes can infer which one is in the role of mixin or based class. This is because they are all parents of the new class like PrintHelloWorld or SaveHelloWorld.

In my attempt to do a policy-based design, it is always possible for a host class to know which classes are their policy because they are its parents. This feature allows me to exploit the Python's Method Resolution Order (MRO) to create compositions between host and policy classes. These compositions are the result of using instantiated host classes as a policy to instantiate another host class, see the example below.

class InputMessage:
    """Generate the message."""
    def run(self):
        return 'hello world'

def AddPrefix(Input):
    """Add a prefix."""
    class _(Input):
        def set_prefix(self, prefix):
            self._prefix = prefix
        def run(self):
            return self._prefix + super().run()
    return _

def AddSuffix(Input):
    """Add a suffix."""
    class _(Input):
        def set_suffix(self, suffix):
            self._suffix = suffix
        def run(self):
            return super().run() + self._suffix
    return _

def PrintOutput(Input):
    """Print message."""
    class _(Input):
        def run(self):
            print(super().run())
    return _

PrintPrefixSuffixMessage = PrintOutput(
    AddSuffix(AddPrefix(InputMessage))
)
message = PrintPrefixSuffixMessage()
message.set_prefix('Victor says: ')
message.set_suffix(' and goodbye!')
message.run()

I am not sure if this difference has any practical implications between these two approaches. At the moment I am just trying to learn what can be expressed with them.

Going down the rabbit hole of recursion

I would like to point out that it is possible to add recursion to the mix. This is reminiscent of some metaprogramming techniques used in C++ that enable computation at compile-time. This has in principle no application in Python that is an interpreted language. Please bear with me for a moment the next impractical example below.

class Identity:
    def run(self, z):
        return z

def MandelbrotSeq(Base, n):
    def Mandelbrot(Base, n):
        if n == 0:
            return Base
        class _(Base):
            def run(self, z):
                z = super().run(z)
                return z**2 + self.c
        return Mandelbrot(_, n-1)
    return Mandelbrot(Base, n).__mro__

M = MandelbrotSeq(Identity, 5)

m = M[0]()
m.c = 1
# print 677 the value at 5th iteration for c=1
print(m.run(0))

m = M[1]()
m.c = 1
# print 26 the value at 4th iteration for c=1
print(m.run(0))

m = M[2]()
m.c = 1
# print 5 the value at 3th iteration for c=1
print(m.run(0))

The factory MandelbrotSeq maps recursively the Mandelbrot sequence as a list of hierarchically connected classes. The first element of M refers to the fifth-generation descendant of the Identity class. This means a call to the run(.) member function of an instance of M[0] will return the 5th value of the sequence. In the same way, I call to run(.) member function of an instance of M[1] will return the 4th value of the sequence.

This is just an extreme example of how to exploit the Python MRO as shown in the earlier section. It is only a toy example, and I was not able to come up with a better idea than something like a fractal just because a recursion was involved. Please do not do this at home!

Design requirements and implementations

Requirement

Design a Python module such users can create classes with new behavior and interfaces by inheriting from a predefined set of classes. This approach provides a large number of classes available to the users based on all possible ways they can be combined into new ones.

This requirement can be implemented using mixin and policy-based implementations.

Mixin implementation

Mixin implementation is defined as follow:

Module classes are divided between mixin and base classes.

Mixin classes implement modifications to the behavior or interface of a base class.

Users create a new class by combining one or more mixin(s) with base classes by inheritance.

class Mixin1Class: """Implementation of some Mixin1 class.""" class Mixin2Class: """Implementation of some Mixin2 class.""" ... class BasedClass: """Implementation of based class.""" # End user creating new class class NewClass(Mixin1Class, [Mixin2Class, ...], BasedClass): pass

References:

Policy-based implementation

Policy-based implementation is defined as follow:

Module classes are divided between policy and host classes.

Policy classes implement modifications to the behavior or interface of the host.

Host classes are defined withing class factories i.e., a function that returns type objects.

Users invoke the creation of new class using class factories.

Policy classes are passed as arguments to the class factory.

Withing the class factory, the host class is created with all the policy classes as its parents.

class Policy1Class: """Implementation of some Policy1 class.""" class Policy2Class: """Implementation of some Policy2 class.""" ... def HostClassFactory(Policy1, Policy2=Policy2Class, ...): """Create a HostClass and return it.""" class HostClass(Policy1, Policy2, ...): """Concrete implementation of the host class.""" return HostClass # End user invoking new class NewClass = HostClassFactory(Policy1Class, [Policy2Class,...])

References:

Comparison between implementations

Here a list of the differences between approaches:

The relationship between classes is not the same affecting the Python's Method Resolution Order or MRO, see figure below.

The mixin's base classes are defined or instantiated when the user creates the new class.

However, policy-based host class definition is delayed until the user calls the factory function.

It is possible to provide default policy classes, but you cannot have mixin default classes.