Using decorators to implement Observer Pattern in Python3

Question

This question is not in general about the observer pattern. It is focused on the use of decorators in that pattern. The question is based on the answer of a similar question.

#!/usr/bin/env python3

class Observable:
    """
        The object that need to be observed. Alternative names are 'Subject'.
        In the most cases it is a data object.
    """
    def __init__(self):
        self._observers = []

    def register_observer(self, callback):
        self._observers.append(callback)
        return callback

    def _broadcast_observers(self, *args, **kwargs):
        for callback in self._observers:
            callback(*args, **kwargs)


class TheData(Observable):
    """
        Example of a data class just for demonstration.
    """
    def __init__(self, data):
        Observable.__init__(self)
        self._data = data

    @property
    def data(self):
        return self._data

    @data.setter
    def data(self, data):
        self._data = data
        self._broadcast_observers()


class TheGUIElement:
    """
        Example of a gui class (Widget) just for demonstration.
        e. g. it could be a text field in GUI.
    """
    def __init__(self, data):
        self._data = data
        #data.register_observer(self._data_updated)
        self._redraw()

    def _redraw(self):
        print('in _redraw(): ' + data.data)

    @Observable.register_observer
    def _data_updated(self, **kwargs):
        """
            This is the callback that is called by the Observable if the
            data changed.
        """
        print('in _data_updated() - kwargs: {}'.format(kwargs))
        self._redraw()


if __name__ == '__main__':
    data = TheData('DATA')
    gui = TheGUIElement(data)

    data.data = 'SECOND DATA'

This code doesn't work because of this error.

Traceback (most recent call last):
  File "./o.py", line 42, in <module>
    class TheGUIElement:
  File "./o.py", line 55, in TheGUIElement
    @Observable.register_observer
TypeError: register_observer() missing 1 required positional argument: 'callback'

It is unclear to me how to use a decorator for to register the observers (e.g. TheGUIElement).

Masterful

Well, like your error message says : "register_observer() missing 1 required positional argument'" : when your decorator is called, it receives only self._data_updated as a parameter, and not self.data. – Leanneleanor

This is because @Observable.register_observer calls register_observer on the Observable class instead of an instance. – Hatching

Answer 1

To register the callback, you need to have an actual object. In your code, how is @Observable.register_observer supposed to find which instance is should register on?

Please drop that Observable thing that's a javaism, cumbersome in python.

Look at this.

#!/usr/bin/env python

class SomeData(object):
    def __init__(self, value):
        self.callbacks = []
        self.foo = value

    def register(self, callback):
        self.callbacks.append(callback)
        return callback

    def notify(self, *args, **kwargs):
        for callback in self.callbacks:
            callback(self, *args, **kwargs)

class SomeGUI(object):
    def redraw(self, obj, key, newvalue):
        print('redrawing %s with value %s' % (self, newvalue))


if __name__ == '__main__':
    my_data = SomeData(42)

    # Register some function using decorator syntax
    @my_data.register
    def print_it(obj, key, value):
        print('Key %s changed to %s' % (key, value))

    # Register the SomeGUI element
    my_gui = SomeGUI()
    my_data.register(my_gui.redraw)

    # Try changing it. Note my_data is dumb for now, notify manually.
    my_data.foo = 10
    my_data.notify("foo", 10)

I intentionally removed automatic notifications to illustrate registration by itself.

Let's add it back. But there is no point using that Observable class. Let's make it lighter, simply defining an event class.

#!/usr/bin/env python3


class Event(object):
    def __init__(self):
        self.callbacks = []

    def notify(self, *args, **kwargs):
        for callback in self.callbacks:
            callback(*args, **kwargs)

    def register(self, callback):
        self.callbacks.append(callback)
        return callback

class SomeData(object):
    def __init__(self, foo):
        self.changed = Event()
        self._foo = foo

    @property
    def foo(self):
        return self._foo

    @foo.setter
    def foo(self, value):
        self._foo = value
        self.changed.notify(self, 'foo', value)

class SomeGUI(object):
    def redraw(self, obj, key, newvalue):
        print('redrawing %s with value %s' % (self, newvalue))


if __name__ == '__main__':
    my_data = SomeData(42)

    # Register some function using decorator syntax
    @my_data.changed.register
    def print_it(obj, key, value):
        print('Key %s changed to %s' % (key, value))

    # Register the SomeGUI element
    my_gui = SomeGUI()
    my_data.changed.register(my_gui.redraw)

    # Try changing it.
    my_data.foo = 10

As you probably noted now, the decorator syntax is useful in those circumstances:

• You have a single registry. Either a singleton or the class itself class are first-order objects, and most are singletons.
• You dynamically define the function and register it as you go.

Now, those manual getters/setters you have are cumbersome as well, if you have many why not factor them out?

#!/usr/bin/env python3


class Event(object):
    def __init__(self):
        self.callbacks = []

    def notify(self, *args, **kwargs):
        for callback in self.callbacks:
            callback(*args, **kwargs)

    def register(self, callback):
        self.callbacks.append(callback)
        return callback

    @classmethod
    def watched_property(cls, event_name, key):
        actual_key = '_%s' % key

        def getter(obj):
            return getattr(obj, actual_key)

        def setter(obj, value):
            event = getattr(obj, event_name)
            setattr(obj, actual_key, value)
            event.notify(obj, key, value)

        return property(fget=getter, fset=setter)


class SomeData(object):
    foo = Event.watched_property('changed', 'foo')

    def __init__(self, foo):
        self.changed = Event()
        self.foo = foo



class SomeGUI(object):
    def redraw(self, obj, key, newvalue):
        print('redrawing %s with value %s' % (self, newvalue))


if __name__ == '__main__':
    my_data = SomeData(42)

    # Register some function using decorator syntax
    @my_data.changed.register
    def print_it(obj, key, value):
        print('Key %s changed to %s' % (key, value))

    # Register the SomeGUI element
    my_gui = SomeGUI()
    my_data.changed.register(my_gui.redraw)

    # Try changing it.
    my_data.foo = 10

For reference, all three programs output the exact same thing:

$ python3 test.py
Key foo changed to 10
redrawing <__main__.SomeGUI object at 0x7f9a90d55fd0> with value 10

Lachrymator

Thank you for investing so much time to answer the question. But your code doesn't fit to my MWE and it is (against my subject and tags) Python2. But of course this two points are not the main reasons why I don't understand your answer. But if you would use my MWE as code basis for your answer it would help me (and other readers) a lot to understand your answer better. – Masterful

What I understand is that you totaly restructred my MWE. I understand your use of Event. But why is print_it a global function? This makes IMO no sense. Your __main__ looks much more complexe then in my MWE. So where is the advantage of your approach? – Masterful

@Masterful This is python3. You can even see at the bottom that I run the code using python3 test.py. – Lachrymator

@Masterful My main is not more complex, I just added the print_it example to show you how it is done with regular functions. Your code missed it and, as it is the prime use for this kind of decorators, I figured I would add one. Other additional lines are comments. – Lachrymator

Why you derive from object? – Masterful

@Masterful (sorry for splitting, I feel it more natural to address points separately). 3) I changed your code structure, because as it is, it does not fit the way this kind of things is done in python. You can of course persist in trying to bend python to look like Java, but you will end up with clumsy code. Overall the solution I suggest uses less code than yours, will feel more natural to any experienced python developer, and is much, much more generic (look at SomeData and SomeGUI, they total 8 lines instead of the 30 you have). – Lachrymator

@Masterful Because that's how you create a class in python3. Not deriving from object creates legacy classes, that are useless but for migrating python 2.1 (and older) code. In fact in python3, even if you omit it, it will silently add it for you and derive from object anyway. – Lachrymator

Let us continue this discussion in chat. – Masterful

@spectras: That is really awesome! I would be grateful if you could elaborate a bit deeper (via a post on your blog maybe...) how everything ties up together. How changing the value of foo triggers the event, What happens when you register a function. I guess it is apparent to seasoned Python programmers but it will help lots of people who dont have that level of experience. In anycase, great work! Many thanks – Acetylate

Answer 2

Even though the thread is kinda old (probably the problem is already solved), I would like to share a solution of mine to the "Decorated Observer Pattern" problem:

https://pypi.org/project/notifyr/

I created a package that implements decorators which add the observer-observed methods/attributes to python classes. I managed to use the package in a Django project too, but with a few adaptations (the .observers attribute is not persisted in the database, so I had to load the list of observers into it every time I expected to notify them).

Here is an implementation example:

Original Code:

class Dog(object):
  def __init__(self, name):
      self.name = name

  def bark(self):
      print('Woof')

  def sleep(self):
      print(self.name, 'is now asleep: ZZzzzzZzzZ...')

class Person(object):
  def __init__(self, name):
      self.name = name

  def educate_dog(self, dog):
      print(self.name + ':','Sleep,', dog.name)
      dog.sleep()

Suppose we want a person to educate a dog every time the animal barks:

from notifyr.agents import observed, observer
from notifyr.functions import target

@observed
class Dog(object):
  def __init__(self, name):
      self.name = name

  @target
  def bark(self):
      print('Woof')

  def sleep(self):
      print(self.name, 'is now asleep: ZZzzzzZzzZ...')

@observer('educate_dog')
class Person(object):
  def __init__(self, name):
      self.name = name

  def educate_dog(self, dog):
      print(self.name + ':','Sleep,', dog.name)
      dog.sleep()

Given the decorated classes, it is possible to achieve the following result:

d = Dog('Tobby')
p = Person('Victor')

d.attach(p) # Victor is now observing Tobby

d.bark()
# Woof
# Victor: Sleep, Tobby
# Tobby is now asleep: ZZzzzzZzzZ...

The package is still very primitive, but it presents a working solution to this type of situation.

Repentant

Answer 3

I was recently looking for something similar and here's what I came up with. It works by intercepting the __setattr__ method -- a useful stunt I plan on keeping in my pocket for later.

def watchableClass(cls):
    """
    Class Decorator!

    * If the class has a "dirty" member variable, then it will be
    automatically set whenever any class value changes
    * If the class has an "onChanged()" method, it will be called
    automatically whenever any class value changes
    * All this only takes place if the value is different from what it was
    that is, if myObject.x is already 10 and you set myObject.x=10 
    nothing happens
    * DOES NOT work with getter/setter functions.  But then, you are
    already in a function, so do what you want!

    EXAMPLE:
        @watchableClass
        class MyClass:
            def __init__(self):
                self.dirty=False
            def onChanged(self):
                print('class has changed')
    """
    if hasattr(cls,'__setattr__'):
        cls.__setattr_unwatched__=cls.__setattr__
        cls.__setattr__=_setObjValueWatchedCascade
    else:
        cls.__setattr__=_setObjValueWatched
    return cls

def _setObjValueWatched(ob,k,v):
    """
    called when an object value is set
    """
    different=not k in ob.__dict__ or ob.__dict__[k]!=v
    if different:
        ob.__dict__[k]=v
        if k not in ('dirty'):
            if hasattr(ob,'dirty'):
                ob.dirty=True
            if hasattr(ob,'onChanged'):
                ob.onChanged()

def _setObjValueWatchedCascade(ob,k,v):
    """
    called when an object value is set
    IF the class had its own __setattr__ member defined!
    """
    different=not k in ob.__dict__ or ob.__dict__[k]!=v
    ob.__setattr_unwatched__(k,v)
    if different:
        if k not in ('dirty'):
            if hasattr(ob,'dirty'):
                ob.dirty=True
            if hasattr(ob,'onChanged'):
                ob.onChanged()

Dumfound

if you add @watchableClass before your class definition, then when any class member is set, it both flags the class as "dirty" (changed) as well as calls its onChanged() method if there is one. – Dumfound