how to make classes with __getattr__ pickable

Question

How can I modify the classes below to make them pickeable?

This question: How to make a class which has __getattr__ properly pickable? is similar but refer to wrong exception in the use of getattr.

This other question seems to provide meaningful insight Why does pickle.dumps call __getattr__?, however it fails to provide an example, and I honestly cannot understand what I am suppose to implement.

import pickle
class Foo(object):
    def __init__(self, dct):
        for key in dct:
            setattr(self, key, dct[key])


class Bar(object):
    def __init__(self, dct):
        for key in dct:
            setattr(self, key, dct[key])

    def __getattr__(self, attr):
        """If attr is not in channel, look in timing_data
        """
        return getattr(self.foo, attr)

if __name__=='__main__':
    dct={'a':1,'b':2,'c':3}
    foo=Foo(dct)
    dct2={'d':1,'e':2,'f':3,'foo':foo}
    bar=Bar(dct2)
    pickle.dump(bar,open('test.pkl','w'))
    bar=pickle.load(open('test.pkl','r'))

Results:

     14         """If attr is not in channel, look in timing_data
     15         """
---> 16         return getattr(self.foo, attr)
     17
     18 if __name__=='__main__':

RuntimeError: maximum recursion depth exceeded while calling a Python object

Speller

Answer 1

The problem here is that your __getattr__ method is poorly implemented. It assumes that self.foo exists. If self.foo doesn't exist, trying to access it ends up calling __getattr__ - which results in infinite recursion:

>>> bar = Bar({})  # no `foo` attribute
>>> bar.x
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "untitled.py", line 19, in __getattr__
    return getattr(self.foo, attr)
  File "untitled.py", line 19, in __getattr__
    return getattr(self.foo, attr)
  File "untitled.py", line 19, in __getattr__
    return getattr(self.foo, attr)
  [Previous line repeated 329 more times]
RecursionError: maximum recursion depth exceeded while calling a Python object

To fix this, you have to throw an AttributeError if no foo attribute exists:

def __getattr__(self, attr):
    """If attr is not in channel, look in timing_data
    """
    if 'foo' not in vars(self):
        raise AttributeError
    return getattr(self.foo, attr)

(I used the vars function to get the object's dict, because it looks nicer than self.__dict__.)

---

Now everything works as expected:

dct={'a':1,'b':2,'c':3}
foo=Foo(dct)
dct2={'d':1,'e':2,'f':3,'foo':foo}
bar=Bar(dct2)
data = pickle.dumps(bar)
bar = pickle.loads(data)
print(vars(bar))
# output:
# {'d': 1, 'e': 2, 'f': 3, 'foo': <__main__.Foo object at 0x7f040fc7e7f0>}

Saida

Adding a little more explanation to the first paragraph: __getattr__ is only called if an attribute cannot be found in "the usual ways", such as checking the object's __dict__. That's why self.foo in __getattr__ is not giving you infinite recursion all the time, but only when self.foo cannot be found in the __dict__. – Tidal