Convenient way to handle deeply nested dictionary in Python

Question

I have a deeply nested dictionary in python thats taking up a lot of room. Is there a way to abbreviate something like this

master_dictionary['sub_categories'][sub_cat_name]['attributes'][attribute_name]['special_type']['nested_children'][child_cat_name][color] = blue

to this for example

nested_child_info[color] = blue

And still have it edit the dictionary ? I hope that makes sense.

Infiltration

That's the quickest way to set just one thing. If you have more data to set, you can grab the lowest common denominator and work from there. – Birck

Answer 1

nested_child_info = master_dictionary['sub_categories'][sub_cat_name]['attributes'][attribute_name]['special_type']['nested_children'][child_cat_name]
nested_child_info[color] = blue

nested_child_info is a reference, so changing its contents is going to change the contents of master_dictionary.

Phlebotomy

Answer 2

Similar to @fferri. You will always have to specify the items in a long list. Get a reference to the final dict using reduce and getitem:

from functools import reduce
from operator import getitem

d = {1:{2:{3:{4:5}}}}

foo = 2
items = [1,foo,3]
result = d
info = reduce(getitem, items, d)


>>> info[4]
5
>>> d
{1: {2: {3: {4: 5}}}}
>>> info[4] = 99
>>> d
{1: {2: {3: {4: 99}}}}

---

I was also playing around with a class but it doesn't seem to have many advantages - except you could customize a key error exception so that the error message would tell you which key at which depth is missing.

class Drilldown:
    def __init__(self, d, path):
        #self.final = reduce(getitem, path, d)
        self.final = d
        for i, item in enumerate(path, 1):
            try:
                self.final = self.final[item]
            except KeyError as e:
                msg = ''.join('[{}]' for _ in range(i))
                msg = msg.format(*path[:i])
                msg = 'The last key in the path "{}" does not exist'.format(msg)
                e.args = [msg]
                raise
    def __call__(self, item):
        return self.final[item]
    def __setitem__(self, item, value):
        self.final[item] = value
    def __getitem__(self, item):
        return self.final[item]
    def __str__(self):
        return str(self.final)
    def __repr__(self):
        return repr(self.final)

>>> z = 19
>>> items = [1,2,z]
>>> q = Drilldown(d,items)
Traceback (most recent call last):
  File "<pyshell#68>", line 1, in <module>
    q = Drilldown(d,items)
  File "C:\pyProjects33\tmp.py", line 32, in __init__
    self.final = self.final[item]
KeyError: 'The last key in the path "[1][2][19]" does not exist'

>>> 
>>> #normal usage
>>> items = [1,2,3]
>>> q = Drilldown(d,items)
>>> d
{1: {2: {3: {4: 5}}}}
>>> q
{4: 5}
>>> q(4)
5
>>> q[4]
5
>>> q[4] += 20
>>> q
{4: 25}
>>> d
{1: {2: {3: {4: 25}}}}
>>> q['foo'] = '99'
>>> q
{4: 25, 'foo': '99'}
>>> d
{1: {2: {3: {4: 25, 'foo': '99'}}}}
>>> 

Concinnate

Answer 3

Yes, you can.

>>> dict1 = {'foo':{'bar':{'baz':0}}}
>>> dict2 = dict1['foo']['bar']
>>> dict2['baz'] = 1
>>> dict1
{'foo': {'bar': {'baz': 1}}} # dict1 has been modified

Splay

Answer 4

Could you do something like this?

thing = {1: {2: {3: {4: {'hello': 'world'}}}}}
a = thing[1]
b = a[2]
c = b[3]
d = c[4]
print(d) # {'hello': 'world'}

Hollyanne

Answer 5

Since dictionaries are mutable, this will actually just happen exactly as you expect:

>>> test = {'outer': 'thing', 'inner': {'thing': 'im the inner thing'}}
>>> inner_thing = test['inner']
>>> inner_thing
{'thing': 'im the inner thing'}
>>> inner_thing['thing'] = 'im something new'
>>> inner_thing
{'thing': 'im something new'}
>>> test
{'outer': 'thing', 'inner': {'thing': 'im something new'}}

This is because mutable objects are passed by reference in python as opposed to being passed as a copy (plenty of great articles on this, so not going to elaborate).

However, it is worth noting that you might not actually want to change the original dict, as it might have undesired effects on other code that uses this variable (it depends on your code base). In this situation, I generally make a copy of the data I need to mutate to avoid side effects:

>>> from copy import deepcopy
>>> test = {'outer': 'thing', 'inner': {'thing': 'im the inner thing'}}
>>> new_test = deepcopy(test)
>>> inner_thing = test['inner']
>>> inner_thing['thing'] = 'im something new'
>>> test
{'outer': 'thing', 'inner': {'thing': 'im something new'}}
>>> new_test
{'outer': 'thing', 'inner': {'thing': 'im the inner thing'}}

Mitzi

Answer 6

If you have some "fixed" keys you could always create a function:

Consider this example:

d = dict(a=dict(sub_categories=dict(b=1)))

def changevalue(value, lvl1, lvl2):
    d[lvl1]['sub_categories'][lvl2] = value

changevalue(2,'a','b')

print(d)

#{'a': {'sub_categories': {'b': 2}}}

In your case you'd want to lift out:

[sub_cat_name], [attribute_name], [child_cat_name], [color] ... maybe

Plumose

Answer 7

You can consider using a NestedDict. Here it is a comparison

# dictionary
master_dictionary['sub_categories'][sub_cat_name]['attributes'][attribute_name]['special_type']['nested_children'][child_cat_name][color] = blue

# NestedDict
key = (
    'sub_categories', 
    sub_cat_name, 
    'attributes', 
    attribute_name, 
    'special_type', 
    'nested_children', 
    child_cat_name, 
    color
)
master_dictionary[key] = blue

You can find NestedDict in ndicts, on PyPi

pip install ndicts

Equipment