Why does `getattr` not support consecutive attribute retrievals?

Question

class A(): pass

a = A()
b = A()

a.b = b
b.c = 1

a.b     # this is b
getattr(a, "b") # so is this

a.b.c   # this is 1   
getattr(a, "b.c") # this raises an AttributeError

It seemed very natural to me to assume the latter. I'm sure there is a good reason for this. What is it?

Phial

Now do this: setattr(a, 'b.c', 2). What should getattr(a, 'b.c') return now? What if there was no c on b before? You are allowed to use a . in attribute names, so you can't expect getattr to be able to traverse over objects like this. – Coze

@MartijnPieters Sadly, it's impossible to accept your great comment as solution/answer and I almost missed it. I will exted that. get/setattr are mapped to magic methods which have single purpose as @ThaneBrimhall said it's the dictionary lookup. For me this is JavaScript related thing where . operator is just syntax sugar for obj['@ttr1but3'] (obj doesn't have to be a mapping). Python' equivalent of this is getattr. Read about __dict__ and try to override __getattribute__ to grasp it yourself. – Cholecystectomy

@JCode: getattr() and setattr() are not mapped to magic methods, not directly. The __getattribute__, __getattr__ and __setattr__ special methods are hooks that, if defined, Python will call for attribute access. getattr() and setattr() are translations of the object.attr expression and object.attr = ... assignment statements that incidentally let you go beyond Python identifiers in the attribute name. And not all Python objects have a __dict__ mapping either, so to say it's a straightforward dictionary lookup is also too simplistic. – Coze

@MartijnPieters Thanks for clarification! I indeed went overboard by forgetting about __slots__. Mentioning magic methods as hooks really improved my understanding. – Cholecystectomy

Answer 1

You can't put a period in the getattr function because getattr is like accessing the dictionary lookup of the object (but is a little bit more complex than that, due to subclassing and other Python implementation details).

If you use the 'dir' function on a, you'll see the dictionary keys that correspond to your object's attributes. In this case, the string "b.c" isn't in the set of dictionary keys.

The only way to do this with getattr is to nest calls:

getattr(getattr(a, "b"), "c")

Luckily, the standard library has a better solution!

import operator
operator.attrgetter("b.c")(a)

Cincture

The OP has realised that... I think he's asking why. Not an unreasonable question given that operator.attrgetter('a.b')(obj) will resolve dotted notation – Dildo

Thanks for pointing that out. Clarified my response to hopefully explain better what I meant. – Cincture

This is a very good and simple solution to a troubling problem. The answers below are not as concise. This answer does not try to implement something that Python already did, which makes it good. – Barone

Note: getattr() is not just as simple as a direct dictionary lookup! Attribute lookup is a bit more complex than this; attribute lookup also allows for descriptor object binding, for example, and there can be a MRO traversal of a class hierarchy (involving multiple dictionaries). – Coze

@MartijnPieters Good point! I'll update the answer to explain the nuance. – Cincture

can't we just have a null coalescing operator and be done with it? Python is > 30 years old. – Whiteness

@ThaneBrimhall the complexity of getattr (beyond a simple dictionary lookup) is NOT an implementation detail. It is very much part of the language specification, and many advanced features (Martijn mentioned descriptors above) depend on it. If anyone wants to look under the hood: snarky.ca/unravelling-attribute-access-in-python – Copyright

Answer 2

Python's built-in reduce function enables the functionality you're looking for. Here's a simple little helper function that will get the job done:

class NoDefaultProvided(object):
    pass

def getattrd(obj, name, default=NoDefaultProvided):
    """
    Same as getattr(), but allows dot notation lookup
    Discussed in:
    http://stackoverflow.com/questions/11975781
    """

    try:
        return reduce(getattr, name.split("."), obj)
    except AttributeError, e:
        if default != NoDefaultProvided:
            return default
        raise

Test proof;

>>> getattrd(int, 'a')
AttributeError: type object 'int' has no attribute 'a'

>>> getattr(int, 'a')
AttributeError: type object 'int' has no attribute 'a'

>>> getattrd(int, 'a', None)
None

>>> getattr(int, 'a', None)
None

>>> getattrd(int, 'a', None)
None

>>> getattrd(int, '__class__.__name__')
type

>>> getattrd(int, '__class__')
<type 'type'>

Brad

This should be the accepted answer imho, the answer from Thane is next to useless. – Clerc

I'm all for a good answer, and this gets the job done. But this answer doesn't give the OP what he was looking for - the reason why it doesn't work. – Cincture

The accepted answer doesn't solve the original problem. Assuming that you are only given the string 'b.c', this is the only answer that solves the problem. – Barbarize

Once again, I'll just say that the point of the question is "why?", not "how can I?". This answer doesn't answer the original question. In addition, this is a very long-winded solution that could otherwise have been done using the standard library: operator.attrgetter('a.b')(obj) – Cincture

Related: Should I flag this answer for suspicious activity or am I just crazy? – Trestle

I found Thane's answer to be extremely useful. It uses a function from the standard library. And in Python 3, you have to import reduce anyway. – Barone

Please don't bring java in python ... gosh ... Thane's answer definitely preferable. – Inflorescence

This had just what I was looking for. Thanks! – Tsushima

Answer 3

I think your confusion arises from the fact that straight dot notation (ex a.b.c) accesses the same parameters as getattr(), but the parsing logic is different. While they both essentially key in to an object's __dict__ attribute, getattr() is not bound to the more stringent requirements on dot-accessible attributes. For instance

setattr(foo, 'Big fat ugly string.  But you can hash it.', 2)

Is valid, since that string just becomes a hash key in foo.__dict__, but

foo.Big fat ugly string.  But you can hash it. = 2

and

foo.'Big fat ugly string.  But you can hash it.' = 2

are syntax errors because now you are asking the interpreter to parse these things as raw code, and that doesn't work.

The flip side of this is that while foo.b.c is equivalent to foo.__dict__['b'].__dict__['c'], getattr(foo, 'b.c') is equivalent to foo.__dict__['b.c']. That's why getattr doesn't work as you are expecting.

Buhler

Answer 4

Because getattr doesn't work that way. getattr gets attribute of a given object (first argument) with a given name (second argument). So your code:

getattr(a, "b.c") # this raises an AttributeError

means: Access "b.c" attribute of object referenced by "a". Obviously your object doesn't have attribute called "b.c".

To get "c" attribute you must use two getattr calls:

getattr(getattr(a, "b"), "c")

Let's unwrap it for better understanding:

b = getattr(a, "b")
c = getattr(b, "c")

Coker

Answer 5

I think the most straight forward way to achieve what you want is to use operator.attrgetter.

>>> import operator
>>> class B():
...   c = 'foo'
... 
>>> class A():
...   b = B()
... 
>>> a = A()
>>> operator.attrgetter('b.c')(a)
'foo'

If the attribute doesn't exist then you'll get an AttributeError

>>> operator.attrgetter('b.d')(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: B instance has no attribute 'd'

Archetype

Answer 6

You can call the multiple getattr without calling a function within function by splitting the dot operators and performing a getattr() for each dot operator

def multi_getattr(self,obj, attr, default = None):
          attributes = attr.split(".")
          for i in attributes:
              try:
                  obj = getattr(obj, i)
              except AttributeError:
                  if default:
                      return default
                  else:
                      raise
          return obj

If suppose you wish to call a.b.c.d you can do it via a.multi_getattr('b.c.d'). This will generalise the operation without worrying about the count of dot operation one has in the string.

Fiendish

Answer 7

What should return getattr('a.b', {'a': None}, 'default-value'}? Should it raise AttributeError or just return 'default-value'? That's why complex keys if introduced in getattr would make it obscure to use.

So, it's more natural to view getattr(..) function as get method of dictionary of object attributes.

Valentijn