getattr and setattr on nested subobjects / chained properties?

Question

I have an object (Person) that has multiple subobjects (Pet, Residence) as properties. I want to be able to dynamically set the properties of these subobjects like so:

class Person(object):
    def __init__(self):
        self.pet = Pet()
        self.residence = Residence()

class Pet(object):
    def __init__(self,name='Fido',species='Dog'):
        self.name = name
        self.species = species

class Residence(object):
    def __init__(self,type='House',sqft=None):
        self.type = type
        self.sqft=sqft


if __name__=='__main__':
    p=Person()
    setattr(p,'pet.name','Sparky')
    setattr(p,'residence.type','Apartment')
    print p.__dict__

Currently I get the wrong output: {'pet': <__main__.Pet object at 0x10c5ec050>, 'residence': <__main__.Residence object at 0x10c5ec0d0>, 'pet.name': 'Sparky', 'residence.type': 'Apartment'}

As you can see, instead of setting the name attribute on the Pet subobject of the Person, a new attribute pet.name is created on the Person.

• I cannot specify person.pet to setattr() because different sub-objects will be set by the same method, which parses some text and fills in the object attributes if/when a relevant key is found.

• Is there a easy/builtin way to accomplish this?

• Or perhaps I need to write a recursive function to parse the string and call getattr() multiple times until the necessary subobject is found and then call setattr() on that found subobject?

Sociable

similar for dict: stackoverflow.com/a/14692747 – Boonie

Answer 1

You could use functools.reduce:

import functools

def rsetattr(obj, attr, val):
    pre, _, post = attr.rpartition('.')
    return setattr(rgetattr(obj, pre) if pre else obj, post, val)

# using wonder's beautiful simplification: https://mcmap.net/q/42050/-getattr-and-setattr-on-nested-subobjects-chained-properties/31174427?noredirect=1#comment86638618_31174427

def rgetattr(obj, attr, *args):
    def _getattr(obj, attr):
        return getattr(obj, attr, *args)
    return functools.reduce(_getattr, [obj] + attr.split('.'))

rgetattr and rsetattr are drop-in replacements for getattr and setattr, which can also handle dotted attr strings.

---

import functools

class Person(object):
    def __init__(self):
        self.pet = Pet()
        self.residence = Residence()

class Pet(object):
    def __init__(self,name='Fido',species='Dog'):
        self.name = name
        self.species = species

class Residence(object):
    def __init__(self,type='House',sqft=None):
        self.type = type
        self.sqft=sqft

def rsetattr(obj, attr, val):
    pre, _, post = attr.rpartition('.')
    return setattr(rgetattr(obj, pre) if pre else obj, post, val)

def rgetattr(obj, attr, *args):
    def _getattr(obj, attr):
        return getattr(obj, attr, *args)
    return functools.reduce(_getattr, [obj] + attr.split('.'))

---

if __name__=='__main__':
    p = Person()
    print(rgetattr(p, 'pet.favorite.color', 'calico'))
    # 'calico'

    try:
        # Without a default argument, `rgetattr`, like `getattr`, raises
        # AttributeError when the dotted attribute is missing
        print(rgetattr(p, 'pet.favorite.color'))
    except AttributeError as err:
        print(err)
        # 'Pet' object has no attribute 'favorite'

    rsetattr(p, 'pet.name', 'Sparky')
    rsetattr(p, 'residence.type', 'Apartment')
    print(p.__dict__)
    print(p.pet.name)
    # Sparky
    print(p.residence.type)
    # Apartment

Caldera

Do you have any idea on how to do a rgetattr that also supports the default parameter? – Untruth

@RedX: The post has been updated to include a default parameter. I wish I could make it a little simpler, but c'est la vie. – Caldera

Will this handle array indexed attributes too? e.g. "pets[0].favorite.color" – Calipee

Hi, Thanks for your inspiration, But I've made a simplified implementation with the same effect: gist.github.com/wonderbeyond/d293e7a2af1de4873f2d757edd580288 – Decree

you can add AttributeError handling there – Geotropism

Can you explain the purpose of having *args be a variable argument? Getattr documentation only appears to accept a single value. – Zrike

Is there a reason to use functools.reduce() instead of an recursive function? – Fye

@shouldsee: Python has a maximum recursion depth. In Python, iteration is generally faster than an equivalent recursive solution. – Caldera

@unutbu, I see. It's probably me too used to writing and reading recursive functions since they are generally more intuitive than looping. – Fye

In case anyone is looking for nested hasattr as well, here is an approach based on the excellent answers to this question. – Zofiazoha

This can further be extended for rhasattr. Check here. – Fotheringhay

note: If you want type safety compliance and mypy not to complain you can use : functools.reduce(lambda obj, obj2: getattr(obj, str(obj2)), "pet.names".split('.'), initial_obj) – Christy

Using operator.attrgetter would work for rgetattr - see: https://mcmap.net/q/42050/-getattr-and-setattr-on-nested-subobjects-chained-properties. Would be great to have a library solution for setting nested attributes (provided intermediate objects exist or are created via a getattr override) – Cadmarr

Answer 2

For an out of the box solution, you can use operator.attrgetter:

from operator import attrgetter
attrgetter(dotted_path)(obj)

Late

I guess I was looking for a corresponding attrsetter so I could both get and set - is there a way to do that with this approach? – Schooling

Note that attrgetter doesn't support a default argument like getattr does. See here. – Stormproof

@JTrana you can use attrgetter to get the parent object and then use setattr on it – Weinrich

@Weinrich Can you explain please what you meant? – Surtax

@Surtax imagine that we want to set the name of the pet's name in a person object. We can get the name as follows attrgetter('pet.name')(p). To change the name we get the parent pet object pet = attrgetter('pet')(p) and then change the name setattr(pet, 'name', 'Arnold'). While it is trivial for this example, it works for any number of hierarchies. – Weinrich

Yep that is the correct solution for getting the attribute - but is there an out of the box solution for setting the dotted attribute? (provided intermediate objects exist or are created via a __getattr__ override) – Cadmarr

Answer 3

For one parent and one child:

if __name__=='__main__':
    p = Person()

    parent, child = 'pet.name'.split('.')
    setattr(getattr(p, parent), child, 'Sparky')

    parent, child = 'residence.type'.split('.')
    setattr(getattr(p, parent), child, 'Sparky')

    print p.__dict__

This is simpler than the other answers for this particular use case.

Sauder

Answer 4

unutbu's answer (https://mcmap.net/q/42050/-getattr-and-setattr-on-nested-subobjects-chained-properties) has a "bug". After getattr() fails and is replaced by default, it continues calling getattr on default.

Example: rgetattr(object(), "nothing.imag", 1) should equal 1 in my opinion, but it returns 0:

• getattr(object(), 'nothing', 1) == 1.
• getattr(1, 'imag', 1) == 0 (since 1 is real and has no complex component).

Solution

I modified rgetattr to return default at the first missing attribute:

import functools

DELIMITER = "."

def rgetattr(obj, path: str, *default):
    """
    :param obj: Object
    :param path: 'attr1.attr2.etc'
    :param default: Optional default value, at any point in the path
    :return: obj.attr1.attr2.etc
    """

    attrs = path.split(DELIMITER)
    try:
        return functools.reduce(getattr, attrs, obj)
    except AttributeError:
        if default:
            return default[0]
        raise

Unbraid

Can you explain why *default is a positional argument when only its first element is ever accessed? – Zrike

Ah I understand now. For anyone else: it is just a way to easily decide if a default value was given. The tuple *default will be empty if no default was set, therefore an if default will return False. If default has an element then we can return it as the default value. Usually this is solved by initializing an object and naming it _DEFAULT or something similar and having the kwarg default=_DEFAULT, but this solution is much simpler, if a bit non-obvious to someone that hasn't seen it before. – Zrike

it was absolutely non-obvious to me either, but I kinda borrowed it from the other answer (but named it default instead). – Unbraid

Answer 5

And a easy to understand three-liner based on jimbo1qaz's answer, reduced to the very limit:

def rgetattr(obj, path, default):
    try:
        return functools.reduce(getattr, path.split(), obj)
    except AttributeError:
        return default

Usage:

>>> class O(object):
...     pass
... o = O()
... o.first = O()
... o.first.second = O()
... o.first.second.third = 42
... rgetattr(o, 'first second third', None)
42

Just keep in mind that "space" is not a typical delimiter for this use case.

Jamille

Answer 6

This should be a

def getNestedAttr(obj,nestedParam):
     next = obj
     for p in nestedParam.split('.'):
         next = getattr(next,p)
     return next


class Issue : pass    
issue = Issue()
issue.status = Issue()
issue.status.name = "Hello"
getattr(issue,'status.name')
'''
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Issue' object has no attribute 'status.name'
'''
getNestedAttr(issue,'status.name')

#'Hello'

simple solution

Awestricken

Answer 7

I made a simple version based on ubntu's answer called magicattr that also works on attrs, lists, and dicts by parsing and walking the ast.

For example, with this class:

class Person:
    settings = {
        'autosave': True,
        'style': {
            'height': 30,
            'width': 200
        },
        'themes': ['light', 'dark']
    }
    def __init__(self, name, age, friends):
        self.name = name
        self.age = age
        self.friends = friends


bob = Person(name="Bob", age=31, friends=[])
jill = Person(name="Jill", age=29, friends=[bob])
jack = Person(name="Jack", age=28, friends=[bob, jill])

You can do this

# Nothing new
assert magicattr.get(bob, 'age') == 31

# Lists
assert magicattr.get(jill, 'friends[0].name') == 'Bob'
assert magicattr.get(jack, 'friends[-1].age') == 29

# Dict lookups
assert magicattr.get(jack, 'settings["style"]["width"]') == 200

# Combination of lookups
assert magicattr.get(jack, 'settings["themes"][-2]') == 'light'
assert magicattr.get(jack, 'friends[-1].settings["themes"][1]') == 'dark'

# Setattr
magicattr.set(bob, 'settings["style"]["width"]', 400)
assert magicattr.get(bob, 'settings["style"]["width"]') == 400

# Nested objects
magicattr.set(bob, 'friends', [jack, jill])
assert magicattr.get(jack, 'friends[0].friends[0]') == jack

magicattr.set(jill, 'friends[0].age', 32)
assert bob.age == 32

It also won't let you/someone call functions or assign a value since it doesn't use eval or allow Assign/Call nodes.

with pytest.raises(ValueError) as e:
    magicattr.get(bob, 'friends = [1,1]')

# Nice try, function calls are not allowed
with pytest.raises(ValueError):
    magicattr.get(bob, 'friends.pop(0)')

Alike

Answer 8

Thanks for the accepted answer above. It was helpful. In case anyone wants to extend the use for hasattr use the code below:

def rhasattr(obj, attr):
    _nested_attrs = attr.split(".")
    _curr_obj = obj
    for _a in _nested_attrs[:-1]:
        if hasattr(_curr_obj, _a):
            _curr_obj = getattr(_curr_obj, _a)
        else:
            return False
    return hasattr(_curr_obj, _nested_attrs[-1])

Fotheringhay

Answer 9

Ok so while typing the question I had an idea of how to do this and it seems to work fine. Here is what I came up with:

def set_attribute(obj, path_string, new_value):
    parts = path_string.split('.')
    final_attribute_index = len(parts)-1
    current_attribute = obj
    i = 0
    for part in parts:
        new_attr = getattr(current_attribute, part, None)
        if current_attribute is None:
            print 'Error %s not found in %s' % (part, current_attribute)
            break
        if i == final_attribute_index:
            setattr(current_attribute, part, new_value)
        current_attribute = new_attr
        i+=1


def get_attribute(obj, path_string):
    parts = path_string.split('.')
    final_attribute_index = len(parts)-1
    current_attribute = obj
    i = 0
    for part in parts:
        new_attr = getattr(current_attribute, part, None)
        if current_attribute is None:
            print 'Error %s not found in %s' % (part, current_attribute)
            return None
        if i == final_attribute_index:
            return getattr(current_attribute, part)
        current_attribute = new_attr
        i += 1

I guess this solves my question, but I am still curious if there is a better way to do this?

I feel like this has to be something pretty common in OOP and python, so I'm surprised gatattr and setattr do not support this natively.

Sociable

The name of the last attribute, final_attribute can occur more than once. For example, p.foo.foo is legal. So the condition part == final_attribute may trigger too soon. – Caldera

Ahh very good point, I changed to checking the index, which should resolve that issue. Thank you for pointing this out! – Sociable

Answer 10

I just love recursive functions

def rgetattr(obj,attr):
    _this_func = rgetattr
    sp = attr.split('.',1)
    if len(sp)==1:
        l,r = sp[0],''
    else:
        l,r = sp

    obj = getattr(obj,l)
    if r:
        obj = _this_func(obj,r)
    return obj

Fye

Answer 11

Here's something similar to ChaimG's answer, but it works with an arbitrary number of cases. However, it only supports get attributes, not setting them.

requested_attr = 'pet.name'
parent = Person()

sub_names = requested_attr.split('.')
sub = None

for sub_name in sub_names:

    try:
        sub = parent.__getattribute__(sub_name)
        parent = sub

    except AttributeError:
        raise Exception("The panel doesn't have an attribute that matches your request!")

pets_name = sub

Savoy

Answer 12

I know this post is pretty old but below code might help some one.

    def getNestedObjectValue(obj={}, attr=""):
    splittedFields = attr.split(".")
    nestedValue = ""
    previousValue = ""
    for field in splittedFields:
        previousValue = nestedValue
        nestedValue = (
            obj.get(field) if previousValue == "" else previousValue.get(field)
        )
    return nestedValue

    print(
    getNestedObjectValue(
        obj={
            "name": "ADASDASD",
            "properties": {"somefield": {"value": "zxczxcxczxcxzc"}},
        },
        attr="properties.somefield.value",
    )
)

Output

PS C:\myprograms\samples> python .\sample.py

zxczxcxczxcxzc

Contessacontest