Python 3 __getattribute__ vs dot access behaviour

Question

I read a bit on Python's object attribute lookup:

Seems pretty straight forward, so I tried it out (python3):

class A:
    def __getattr__(self, attr):
        return (1,2,3)

a = A()

a.foobar #returns (1,2,3) as expected
a.__getattribute__('foobar') # raises AttributeError

My question is, aren't the two supposed to be identical?

Why does the second one raise an attribute error?

So apparently the answer is that the logic for a.foobar IS different from the logic for a.__getattribute("foobar"). According to the data model: a.foobar calls a.__getattribute("foobar") and if it raises an AttributeError, it calls a.-__getattr__('foobar')

So it seems the article has a mistake in their diagram. Is this correct?

And another question: Where does the real logic for a.foobar sit? I thought it was in __getattribute__ but apparently not entirely.

Edit:
Not a duplicate of Difference between __getattr__ and __getattribute__.

I am asking here what is the different between object.foo and object.__getattribute__("foo").
This is different from __getattr__ vs __getatribute__ which is trivial...

Latoyialatreece

Refer to official doc first: docs.python.org/3.5/reference/… – Phosphorite

Change a.__getattribute__('foobar') to a.__getattr__('foobar') and see what happens. – Boothman

Not interested in that... __getattribute__ is supposed to eventually call __getattr__. a.foobar is supposed to call __getattribute__ – Latoyialatreece

Why? It doesn't exist so __getattr__ is invoked instead. – Convexity

@Phosphorite the data model spec is unclear. Does this mean that a.foo is implemented like this: try a.__getattribute__ except AtributeError: a.__getattr__ ? – Latoyialatreece

@dellar More or less. __getattr__ is only tried if it exists, and there are special exceptions when magic methods are used. For instance, len bypasses both __getattribute__ and __getattr__ to retrieve __len__. – Projection

@Latoyialatreece the __getattribute__ is a decriptor, defined in one or more base classes (or in object if not elsewhere). By default it invokes __getattr__ for unknown (not present in __dict__ or nondefined as __slots__ attrs). So usually you need to redefine __getattr__ to intercept unknown attributes, and __getattribute__ gives you total control (regulary not needed). Something like it – Phosphorite

@thodnev: __getattribute__ doesn't invoke __getattr__; the fallback to __getattr__ is done separately. The fact that this is not __getattribute__'s responsibility is probably the source of the questioner's confusion. – Scopas

@Scopas If the class also defines __getattr__(), the latter will not be called unless __getattribute__() either calls it explicitly or raises an AttributeError – Phosphorite

@Scopas This is consistent with what i tried. Does this mean the diagram on the article is mistaken? it shows __getattribute__calling __getattr__ – Latoyialatreece

@thodnev: That last bit about "or raises an AttributeError" is key. If __getattribute__ raises an AttributeError, the attribute lookup process will try __getattr__ next. – Scopas

Where is all of this logic implemented? i thought __getattribute__ was in charge of ALL the lookup process... – Latoyialatreece

@Scopas Do you guys believe this is a really a duplicate? Why was it marked as such? how can i remove the duplicate? i think its a legitimate question... – Latoyialatreece

@Convexity please take a look. I do not think this is a duplicate... – Latoyialatreece

Answer 1

It's easy to get the impression that __getattribute__ is responsible for more than it really is. thing.attr doesn't directly translate to thing.__getattribute__('attr'), and __getattribute__ is not responsible for calling __getattr__.

The fallback to __getattr__ happens in the part of the attribute access machinery that lies outside __getattribute__. The attribute lookup process works like this:

• Find the __getattribute__ method through a direct search of the object's type's MRO, bypassing the regular attribute lookup process.
• Try __getattribute__.
  • If __getattribute__ returned something, the attribute lookup process is complete, and that's the attribute value.
  • If __getattribute__ raised a non-AttributeError, the attribute lookup process is complete, and the exception propagates out of the lookup.
  • Otherwise, __getattribute__ raised an AttributeError. The lookup continues.
• Find the __getattr__ method the same way we found __getattribute__.
  • If there is no __getattr__, the attribute lookup process is complete, and the AttributeError from __getattribute__ propagates.
• Try __getattr__, and return or raise whatever __getattr__ returns or raises.

At least, in terms of the language semantics, it works like that. In terms of the low-level implementation, some of these steps may be optimized out in cases where they're unnecessary, and there are C hooks like tp_getattro that I haven't described. You don't need to worry about that kind of thing unless you want to dive into the CPython interpreter source code.

Scopas

TL;DR __getattribute__ bypasses the regular attribute lookup only if it is defined and is not intended to call __getattr__. Calling __getattribute__ directly on a class which does not define it, falls back to object.__getattribute__ or another base class and thus raises (probably) an AttributeError – Luau