Understanding the difference between __getattr__ and __getattribute__

Question

I am trying to understand the difference between __getattr__ and __getattribute__, however, I am failing at it.

The answer to the Stack Overflow question Difference between __getattr__ vs __getattribute__ says:

__getattribute__ is invoked before looking at the actual attributes on the object, and so can be tricky to implement correctly. You can end up in infinite recursions very easily.

I have absolutely no idea what that means.

Then it goes on to say:

You almost certainly want __getattr__.

Why?

I read that if __getattribute__ fails, __getattr__ is called. So why are there two different methods doing the same thing? If my code implements the new style classes, what should I use?

I am looking for some code examples to clear this question. I have Googled to best of my ability, but the answers that I found don't discuss the problem thoroughly.

If there is any documentation, I am ready to read that.

Transpire

if it helps, from the docs "In order to avoid infinite recursion in this method, its implementation should always call the base class method with the same name to access any attributes it needs, for example, object.__getattribute__(self, name)." [docs.python.org/2/reference/… – Anabolite

Possible duplicate of Difference between __getattr__ vs __getattribute__ – Humanist

Answer 1

Some basics first.

With objects, you need to deal with their attributes. Ordinarily, we do instance.attribute. Sometimes we need more control (when we do not know the name of the attribute in advance).

For example, instance.attribute would become getattr(instance, attribute_name). Using this model, we can get the attribute by supplying the attribute_name as a string.

Use of __getattr__

You can also tell a class how to deal with attributes which it doesn't explicitly manage and do that via __getattr__ method.

Python will call this method whenever you request an attribute that hasn't already been defined, so you can define what to do with it.

A classic use case:

class A(dict):
    def __getattr__(self, name):
       return self[name]
a = A()
# Now a.somekey will give a['somekey']

Caveats and use of __getattribute__

If you need to catch every attribute regardless whether it exists or not, use __getattribute__ instead. The difference is that __getattr__ only gets called for attributes that don't actually exist. If you set an attribute directly, referencing that attribute will retrieve it without calling __getattr__.

__getattribute__ is called all the times.

Unaesthetic

" For example, instance.attribute would become getattr(instance, attribute_name). " . Shouldn't it be __getattribute__(instance, attribute_name) ? – Inexpugnable

@Md.AbuNafeeIbnaZahid getattr is a built-in function. getattr(foo, 'bar') is equivalent to foo.bar. – Gamez

worth noting that __getattr__ only gets called if it's defined, as it's not inherited from object – Chelonian

Answer 2

__getattribute__ is called whenever an attribute access occurs.

class Foo(object):
    def __init__(self, a):
        self.a = 1

    def __getattribute__(self, attr):
        try:
            return self.__dict__[attr]
        except KeyError:
            return 'default'
f = Foo(1)
f.a

This will cause infinite recursion. The culprit here is the line return self.__dict__[attr]. Let's pretend (It's close enough to the truth) that all attributes are stored in self.__dict__ and available by their name. The line

f.a

attempts to access the a attribute of f. This calls f.__getattribute__('a'). __getattribute__ then tries to load self.__dict__. __dict__ is an attribute of self == f and so python calls f.__getattribute__('__dict__') which again tries to access the attribute '__dict__'. This is infinite recursion.

If __getattr__ had been used instead then

1. It never would have run because f has an a attribute.
2. If it had run, (let's say that you asked for f.b) then it would not have been called to find __dict__ because it's already there and __getattr__ is invoked only if all other methods of finding the attribute have failed.

The 'correct' way to write the above class using __getattribute__ is

class Foo(object):
    # Same __init__

    def __getattribute__(self, attr):
        return super().__getattribute__(attr)

super().__getattribute__(attr) binds the __getattribute__ method of the 'nearest' superclass (formally, the next class in the class's Method Resolution Order, or MRO) to the current object self and then calls it and lets that do the work.

All of this trouble is avoided by using __getattr__ which lets Python do it's normal thing until an attribute isn't found. At that point, Python hands control over to your __getattr__ method and lets it come up with something.

It's also worth noting that you can run into infinite recursion with __getattr__.

class Foo(object):
    def __getattr__(self, attr):
        return self.attr

I'll leave that one as an exercise.

Emcee

When you call super(Foo, self), is that right that you are explicitly grabbing the nearest parent class of Foo and then passing in instance self as the second argument? Could you have done super() instead of super(Foo, self) since Python 3 uses Method Resolution Order to implicitly figure out self's most relevant parent class (which is very useful when using multiple inheritance) ?? – Hyo

@E.Kaufman, bro the answer was posted on 2010, over ten years by the time you made such an ignorant comment. – Jurywoman

Edited the answer to follow PEP 3135; Use super() instead of the super(Foo, self). – Discipline

Answer 3

I think the other answers have done a great job of explaining the difference between __getattr__ and __getattribute__, but one thing that might not be clear is why you would want to use __getattribute__. The cool thing about __getattribute__ is that it essentially allows you to overload the dot when accessing a class. This allows you to customize how attributes are accessed at a low level. For instance, suppose I want to define a class where all methods that only take a self argument are treated as properties:

# prop.py
import inspect

class PropClass(object):
    def __getattribute__(self, attr):
        val = super().__getattribute__(attr)
        if callable(val):
            argcount = len(inspect.getargspec(val).args)
            # Account for self
            if argcount == 1:
                return val()
            else:
                return val
        else:
            return val

And from the interactive interpreter:

>>> import prop
>>> class A(prop.PropClass):
...     def f(self):
...             return 1
... 
>>> a = A()
>>> a.f
1

Of course this is a silly example and you probably wouldn't ever want to do this, but it shows you the power you can get from overriding __getattribute__.

Leveroni

Answer 4

I have gone through other's excellent explanation. However, I found a simple answer from this blog Python Magic Methods and __getattr__. All the following are from there.

Using the __getattr__ magic method, we can intercept that inexistent attribute lookup and do something so it doesn’t fail:

class Dummy(object):

    def __getattr__(self, attr):
        return attr.upper()

d = Dummy()
d.does_not_exist # 'DOES_NOT_EXIST'
d.what_about_this_one  # 'WHAT_ABOUT_THIS_ONE'

But if the attribute does exist, __getattr__ won’t be invoked:

class Dummy(object):

    def __getattr__(self, attr):
        return attr.upper()

d = Dummy()
d.value = "Python"
print(d.value)  # "Python"

__getattribute__ is similar to __getattr__, with the important difference that __getattribute__ will intercept EVERY attribute lookup, doesn’t matter if the attribute exists or not.

class Dummy(object):

    def __getattribute__(self, attr):
        return 'YOU SEE ME?'

d = Dummy()
d.value = "Python"
print(d.value)  # "YOU SEE ME?"

In that example, the d object already has an attribute value. But when we try to access it, we don’t get the original expected value (“Python”); we’re just getting whatever __getattribute__ returned. It means that we’ve virtually lost the value attribute; it has become “unreachable”.

Subirrigate