Using __setattr__ and __getattr__ for delegation with __slots__ without triggering infinite recursion

Question

class A:
    __slots__ = ("a",)
    def __init__(self) -> None:
        self.a = 1

class B1:
    __slots__ = ("b",)
    def __init__(self, b) -> None:
        self.b = b

    def __getattr__(self, k):
        return getattr(self.b, k)

    def __setattr__(self, k, v):
        setattr(self.b, k, v)

class B2:
    __slots__ = ("b",)
    def __init__(self, b) -> None:
        self.b = b

    def __getattr__(self, k):
        return getattr(super().__getattr__("b"), k)

    def __setattr__(self, k, v):
        setattr(super().__getattr__("b"), k, v)

class B3:
    __slots__ = ("b",)
    def __init__(self, b) -> None:
        self.b = b

    def __getattr__(self, k):
        return getattr(getattr(super(), "b"), k)

    def __setattr__(self, k, v):
        setattr(getattr(super(), "b"), k, v)

a = A()
b = B1(a)
print(b.a) # RecursionError: maximum recursion depth exceeded

b = B2(a)
print(b.a) # AttributeError: 'super' object has no attribute '__getattr__'

b = B3(a)
print(b.a) # AttributeError: 'super' object has no attribute 'b'

Metagalaxy

BTW, I have only created this question in order to share my solution, because I have searched SO and found no question having the same setting as mine (__slots__ are not very popular among python programmers) and had to do a bit of research myself to figure out how to fix it. – Metagalaxy

Answer 1

A more proper way is to check if the attribute name is in any of the available __slots__ up the class hierarchy before delegating:

class BCorrect(object):
    __slots__ = ('b',)

    def __init__(self, b) -> None:
        self.b = b

    def _in_slots(self, attr) -> bool:
        for cls in type(self).__mro__:
            if attr in getattr(cls, '__slots__', []):
                return True
        return False

    def __getattr__(self, attr):
        if self._in_slots(attr):
            return object.__getattr__(self, attr)
        return getattr(self.b, attr)

    def __setattr__(self, attr, value):
        if self._in_slots(attr):
            object.__setattr__(self, attr, value)
            return
        setattr(self.b, attr, value)

This has the advantages that it does not break inheritance and does not need any magic in __init__.

Leukemia

Instead it has magic within __getattr__ and __setattr__ and more python code within them with an additional function call, traversal of the class hierarchy implemented in python and searching within __slots__ (they probably can be made a set to optimize that a bit). I haven't measured performance impact of this, but suspect that this implementation has bigger overhead. – Metagalaxy

It isn't about overhead. The solution you posted and accepted will not work if you inherit from a class that defines __slots__ or if another class inherits yours. – Leukemia

It definitely does not work. The only reason it worked for you is because you repeated the same trick in all your child classes, which is unrealistic. You will forget at some point, or other devs will simply not know that they must initialize their instances the same way you did. Likewise, parent classes would not know about the trick in your __init__ and would not be coded to work with it. Try for yourself: ideone.com/3pIpWi – Leukemia

You are right. I have marked your solution as an answer, but would likely continue to use a my one in my projects (I guess because of its lower overhead, but I have not measured). When I implement autodelegation, all the classes in the inheritance hierarchy are likely to be controlled by me. Sorry for the late qnswer, just noticed that. – Metagalaxy

Answer 2

Python __slots__ is just a sugar for auto-generated descriptors. Calling descriptors is implemented within __setattr__ and __getattr__ (or __*attribute__, I haven't dug deep) of object. The most importantly, we have overridden the default __setattr__ and as a result, were unable to initialize the value using dot notation within the ctor. Since the value of the slotted variable is not yet initialized, our __setattr__ causes access to __getattr__ (an incorrect behaviour by itself!), and __getattr__ needs the slotted variable itself, so - infinite recursion.

For non-__slots__ classes it is worked around using __dict__. We cannot use __dict__ for it because we don't have them in __slots__ classes.

The docs says that __slots__ are implemented as descriptors. Descriptors are special objects with magic methods, set into class the same way static methods and props are set (BTW classmethod and staticmethod also construct descriptors), usually acting not on the object itself, but on its parent class.

So, to initialize the value correctly, we should call the descriptor method explicitly

class BCorrect:
    __slots__ = ("b",)
    def __init__(self, b) -> None:
        self.__class__.b.__set__(self, b)

    def __getattr__(self, k):
        return getattr(self.b, k)

    def __setattr__(self, k, v):
        setattr(self.b, k, v)

And then everything works as intended:

b = BCorrect(a)
print(b.a)  # 1
b.a = 2
print(a.a)  # 2

https://www.ideone.com/3yfpbv

Metagalaxy

Answer 3

I think it is a good usage of super. It follows the principle of object.__setattr__ and works if you inherit the class and also if that subclass uses slots too.

class A:
    __slots__ = ('a',)

    def __init__(self):
        self.a = 1
        return None
    def __getattr__(self, name):
        return getattr(self, name)
    
    def __setattr__(self, name, value):
        return super().__setattr__(name, value)
    pass

Poulenc