Avoiding infinite loops in __getattribute__ [duplicate]

Question

The method __getattribute__ needs to be written carefully in order to avoid the infinite loop. For example:

class A:
    def __init__(self):
        self.x = 100

    def __getattribute__(self, x):
        return self.x

>>> a = A()
>>> a.x    # infinite looop
RuntimeError: maximum recursion depth exceeded while calling a Python object


class B:
    def __init__(self):
        self.x = 100

    def __getattribute__(self, x):
        return self.__dict__[x]

>>> b = B()
>>> b.x    # infinite looop
RuntimeError: maximum recursion depth exceeded while calling a Python object

Hence we need to write the method in this way:

class C:
    def __init__(self):
        self.x = 100

    def __getattribute__(self, x):
        # 1. error
        # AttributeError: type object 'object' has no attribute '__getattr__'
        # return object.__getattr__(self, x)
        
        # 2. works
        return object.__getattribute__(self, x)
        
        # 3. works too
        # return super().__getattribute__(x)

My question is why does object.__getattribute__ method work? From where does object get the __getattribute__ method? And if object does not have any __getattribute__, then we are just calling the same method on class C but via the super class. Why, then calling the method via super class does not result in an infinite loop?

Susy

Are you sure you need __getattribute__ and not __getattr__? – Monoplane

Yes I am, because I need to intercept ALL the attribute fetches in my class. But even if I didn't, I would still want to know the full details of why this is so. – Susy

Well, either you have to intercept all attribute access or you don't :-) – Monoplane

@MartijnPieters -- I agree. My main confusion is WHY calling the SAME method from object does not result in infinite recursion. – Susy

Why do you say that "object does not have any __getattribute__"? a = object(); dir(a) and you'll see it listed ... – Ambulance

Answer 1

You seem to be under the impression that your implementation of __getattribute__ is merely a hook, that if you provide it Python will call it, and otherwise the interpreter will do its normal magic directly.

That is not correct. When python looks up attributes on instances, __getattribute__ is the main entry for all attribute access, and object provides the default implementation. Your implementation is thus overriding the original, and if your implementation provides no alternative means of returning attributes it fails. You cannot use attribute access in that method, since all attribute access to the instance (self) is channelled again through type(self).__getattribute__(self, attr).

The best way around this is by calling the overridden original again. That's where super(C, self).__getattribute__(attr) comes in; you are asking the next class in the class-resolution order to take care of the attribute access for you.

Alternatively, you can call the unbound object.__getattribute__() method directly. The C implementation of this method is the final stop for attribute access (it has direct access to __dict__ and is thus not bound to the same limitations).

Note that super() returns a proxy object that'll look up whatever method can be found next in the method-resolution ordered base classes. If no such method exists, it'll fail with an attribute error. It will never call the original method. Thus Foo.bar() looking up super(Foo, self).bar will either be a base-class implementation or an attribute error, never Foo.bar itself.

Monoplane

Answer 2

When you do this:

    return object.__getattribute__(self, x)

you are calling a specific function -- the one defined in the object class, and not the one defined in A, so there is no recursion.

When you do this:

    return self.x

you are letting python choose which function to call, and it calls the one from A, and you have an infinite recursion.

Dictatorship

There is NO such function defined on the object class. If there were a function defined there, then object would also define __getattr__ (which, as the #1 error shows it does not). So, then why would object define just the __getattribute__ method? – Susy

@greengit: why wouldn't it? I see <slot wrapper '__getattribute__' of 'object' objects> when I access object.__getattribute__. – Monoplane

So you're saying object only defines __getattribte__ and NOT __getattr__? As far as I know, the object class's implementation of all these methods are no-op. Correct me if I am wrong somewhere. – Susy

@greengit: object only implements __getattribute__. That method will call __getattr__ if it is available and the attribute wasn't found elsewhere. It is decidedly not a no-op. – Monoplane

@MartijnPieters -- Looks like you are correct. A simple dir(object) shows that object ONLY defines a __getattribute__, and NO __getattr__. I thouught object had no-op implementations for ALL the methods. – Susy

@greengit: Nope, object.__getattribute__ is the main access point for object attribute access. When you define that on a subclass, you are overriding it and to get the normal behaviour again you need to call the overridden implementation. – Monoplane

"I thought object had no-op implementations"? Why in the world...? How would things even work if the root object's implementations of special methods were no-ops? What would you be overriding when you wrote your own? How would those overrides access the base functionality if the base methods didn't do anything? It doesn't even make sense. – Revolver

Answer 3

Write it like this (C inherits from object):

class C(object):
   def __init__(self):
       self.x = 100

   def __getattribute__(self, x):
       return object.__getattribute__(self, x)

Now you see why object.__getattribute__(self, x) works - you are calling parent object.

Bryanbryana