Setting a value in a nested Python dictionary given a list of indices and value

Question

I'm trying to programmatically set a value in a dictionary, potentially nested, given a list of indices and a value.

So for example, let's say my list of indices is:

['person', 'address', 'city']

and the value is

'New York'

I want as a result a dictionary object like:

{ 'Person': { 'address': { 'city': 'New York' } }

Basically, the list represents a 'path' into a nested dictionary.

I think I can construct the dictionary itself, but where I'm stumbling is how to set the value. Obviously if I was just writing code for this manually it would be:

dict['Person']['address']['city'] = 'New York'

But how do I index into the dictionary and set the value like that programmatically if I just have a list of the indices and the value?

Python

Manteltree

What's wrong with dict[x][y][z] = value? Where do you get value from? What's with the weird format? – Inexpungible

@Inexpungible It's wrong because he may have variable-length number of nested keys. – Arapaho

You may be able to avoid nesting by using tuple keys: d[tuple(da_list)] = value – Raynold

For greater context, I have to take numerous strings in the format 'key1:key2:key3=value' and turn this into a nested dictionary, where key1 is at the first level, key2 at the second etc. The strings might have one key, two keys, three keys. So a bit of string splitting gets me a list of keys, and a value and where I go from there is the next question... – Manteltree

The question is: "why do you have to turn this into a nested dictionary?" Are you sure you really need a nested dictionary? Things like this can often be coded better with tuple keys, or other data structures. – Arapaho

Ultimately the constructed object will be spit out as JSON...where the hierarchy is necessary. But if there's a better way to go from my strings to a stringified JSON object I'm very open to it :) I may be over-engineering the solution!? – Manteltree

What's wrong with just d["key1:key2:key3"] = value? – Gambill

If the problem is the json conversion, then you may be able to use tuple-keys internally and only convert to a nested structure when you have to produce the json. @Gambill I imagine that the program that receives the data expects something nested as shown in the question, so using a string would not produce an usable output anyway(even if valid json). – Arapaho

@Bakuriu. Yes, my comment was a bit rhetorical: the OP hasn't fully defined his real problem. – Gambill

Answer 1

Something like this could help:

def nested_set(dic, keys, value):
    for key in keys[:-1]:
        dic = dic.setdefault(key, {})
    dic[keys[-1]] = value

And you can use it like this:

>>> d = {}
>>> nested_set(d, ['person', 'address', 'city'], 'New York')
>>> d
{'person': {'address': {'city': 'New York'}}}

Arapaho

Any clue please why this works? (I've tried it and works) And why a pure for loop does not? – Err

@Err This works because inside the for look you re-assign the dic variable to point to the inner dictionary. In the example, when nested_set is set, initially you have dic == d. Then the for loop performs the call dic = dic.setdefault("person", {}) which adds the pair "person": {} to d and also returns the referenced {}. Now dic is d["person"], so the next loop sets the key address in the inner dictionary, setting dic to point to the inner-inner dictionary. Etc. The last key is treated in a special way because you don't want to create dictionaries anymore. – Arapaho

@Arapaho It will break for the attached case as it doesn't replace for a pre-assigned key. Refer my implementation for the correct behavior imgur.com/a/pST93yQ – Lewiss

@nehemiah That's a different behaviour than what the question required. As you can see it adds quite a bit of complexity in the implementation. – Arapaho

Answer 2

I took the freedom to extend the code from the answer of Bakuriu. Therefore upvotes on this are optional, as his code is in and of itself a witty solution, which I wouldn't have thought of.

def nested_set(dic, keys, value, create_missing=True):
    d = dic
    for key in keys[:-1]:
        if key in d:
            d = d[key]
        elif create_missing:
            d = d.setdefault(key, {})
        else:
            return dic
    if keys[-1] in d or create_missing:
        d[keys[-1]] = value
    return dic

When setting create_missing to True, you're making sure to only set already existing values:

# Trying to set a value of a nonexistent key DOES NOT create a new value
print(nested_set({"A": {"B": 1}}, ["A", "8"], 2, False))
>>> {'A': {'B': 1}}

# Trying to set a value of an existent key DOES create a new value
print(nested_set({"A": {"B": 1}}, ["A", "8"], 2, True))
>>> {'A': {'B': 1, '8': 2}}

# Set the value of an existing key
print(nested_set({"A": {"B": 1}}, ["A", "B"], 2))
>>> {'A': {'B': 2}}

Mcconaghy

Answer 3

Here's another option:

from collections import defaultdict
recursivedict = lambda: defaultdict(recursivedict)
mydict = recursivedict()

I originally got this from here: Set nested dict value and create intermediate keys.

It is quite clever and elegant if you ask me.

Autum

Answer 4

First off, you probably want to look at setdefault.

As a function I'd write it as

def get_leaf_dict(dct, key_list):
    res=dct
    for key in key_list:
        res=res.setdefault(key, {})
    return res

This would be used as:

get_leaf_dict( dict, ['Person', 'address', 'city']) = 'New York'

This could be cleaned up with error handling and such. Also using *args rather than a single key-list argument might be nice; but the idea is that you can iterate over the keys, pulling up the appropriate dictionary at each level.

Fruitcake

And thank you for this...Bakuriu's answer is slightly more comprehensive/concise, but you were on the same track and I appreciate how quickly you replied. – Manteltree

I get a SyntaxError: can't assign to function call in Python 3 for get_leaf_dict( dict, ['Person', 'address', 'city']) = 'New York'. – Distinctive

Answer 5

Here is my simple solution: just write

terms = ['person', 'address', 'city'] 
result = nested_dict(3, str)
result[terms] = 'New York'  # as easy as it can be

You can even do:

terms = ['John', 'Tinkoff', '1094535332']  # account in Tinkoff Bank
result = nested_dict(3, float)
result[terms] += 2375.30

Now the backstage:

from collections import defaultdict


class nesteddict(defaultdict):
    def __getitem__(self, key):
        if isinstance(key, list):
            d = self
            for i in key:
                d = defaultdict.__getitem__(d, i)
            return d
        else:
            return defaultdict.__getitem__(self, key)
    def __setitem__(self, key, value):
        if isinstance(key, list):
            d = self[key[:-1]]
            defaultdict.__setitem__(d, key[-1], value)
        else:
            defaultdict.__setitem__(self, key, value)


def nested_dict(n, type):
    if n == 1:
        return nesteddict(type)
    else:
        return nesteddict(lambda: nested_dict(n-1, type))

Blaspheme

Answer 6

Use these pair of methods

def gattr(d, *attrs):
    """
    This method receives a dict and list of attributes to return the innermost value of the give dict
    """
    try:
        for at in attrs:
            d = d[at]
        return d
    except:
        return None


def sattr(d, *attrs):
    """
    Adds "val" to dict in the hierarchy mentioned via *attrs
    For ex:
    sattr(animals, "cat", "leg","fingers", 4) is equivalent to animals["cat"]["leg"]["fingers"]=4
    This method creates necessary objects until it reaches the final depth
    This behaviour is also known as autovivification and plenty of implementation are around
    This implementation addresses the corner case of replacing existing primitives
    https://gist.github.com/hrldcpr/2012250#gistcomment-1779319
    """
    for attr in attrs[:-2]:
        # If such key is not found or the value is primitive supply an empty dict
        if d.get(attr) is None or isinstance(d.get(attr), dict):
            d[attr] = {}
        d = d[attr]
    d[attrs[-2]] = attrs[-1]

Lewiss

Don't use a bare except:, it should never appear in code. You want to use except KeyError instead. Also your solution breaks if someones uses a different mapping type than dict. You may want to use abc.MutableMapping or abc.Mapping – Arapaho

Answer 7

The dotty_dict library for Python 3 can do this. See documentation, Dotty Dict for more clarity.

from dotty_dict import dotty

dot = dotty()
string = '.'.join(['person', 'address', 'city'])
dot[string] = 'New York'

print(dot)

Output:

{'person': {'address': {'city': 'New York'}}}

Simpson

Answer 8

Here's a variant of Bakuriu's answer that doesn't rely on a separate function:

keys = ['Person', 'address', 'city']
value = 'New York'

nested_dict = {}

# Build nested dictionary up until 2nd to last key
# (Effectively nested_dict['Person']['address'] = {})
sub_dict = nested_dict
for key_ind, key in enumerate(keys[:-1]):
    if not key_ind:
        # Point to newly added piece of dictionary
        sub_dict = nested_dict.setdefault(key, {})
    else:
        # Point to newly added piece of sub-dictionary
        # that is also added to original dictionary
        sub_dict = sub_dict.setdefault(key, {})
# Add value to last key of nested structure of keys
# (Effectively nested_dict['Person']['address']['city'] = value)
sub_dict[keys[-1]] = value

print(nested_dict)

>>> {'Person': {'address': {'city': 'New York'}}}

Tao

Answer 9

This is a pretty good use case for a recursive function. So you can do something like this:

def parse(l: list, v: str) -> dict:
    copy = dict()
    k, *s = l
    if len(s) > 0:
        copy[k] = parse(s, v)
    else:
        copy[k] = v
    return copy

This effectively pops off the first value of the passed list l as a key for the dict copy that we initialize, then runs the remaining list through the same function, creating a new key under that key until there's nothing left in the list, whereupon it assigns the last value to the v param.

Size

Answer 10

you can write your own helper class to make your code clear.
dict_helper.py:

from collections import UserDict

# reference: https://realpython.com/inherit-python-dict/

class DictHelper(UserDict):
    """
    reference: https://mcmap.net/q/42018/-elegant-way-to-check-if-a-nested-key-exists-in-a-dict
    """

    def has_key(self, path: str | list):
        keys = path.split('.') if isinstance(path, str) else path

        val = self.data
        for key in keys:
            if not key in val:
                return False
            else:
                val = val[key]
        return True

    # Throwing in this approach for nested get for the heck of it...
    def get_key(self, path: str | list, default=None):
        keys = path.split('.') if isinstance(path, str) else path

        val = self.data
        for key in keys:
            if key in val:
                val = val[key]
            else:
                return default
        return val

    def set_key(self, path: str | list, value):
        keys = path.split('.') if isinstance(path, str) else path

        val = self.data
        for key in keys[:-1]:
            val.setdefault(key, {})
            val = val[key]
        val[keys[-1]] = value

main.py:

from dict_helper import DictHelper

# way1
d = DictHelper({})
d.set_key(['person', 'address', 'city'], 'New York')
print(d)

# or way2
d = DictHelper({})
d.set_key('person.address.city', 'New York')
print(d) 
# both way has the same result
# output={'person': {'address': {'city': 'New York'}}} 

you can also use your own helper functions and helper classes in future projects.

new update 17 Feb 2024:

there is a very good library exists that can handle your class easier. just install it by below command:

pip install "python-benedict[all or io for this project]"

now you can define this class very easily:

from collections import UserDict
from benedict import benedict


class DictHelper(UserDict):

def __init__(self, d: dict = None):
    super().__init__(d)
    self.d = benedict().from_json(self.data)

@staticmethod
def __keypath__(path: str | list):
    return path if isinstance(path, str) else '.'.join(path)

def has_key(self, path: str | list):
    return self.__keypath__(path) in self.d

def get_key(self, path: str | list):
    return self.d[self.__keypath__(path)]

def set_key(self, path: str | list, value=None):
    self.d[self.__keypath__(path)] = value

Mollescent

Answer 11

This is much easier in Perl:

my %hash;
$hash{"aaa"}{"bbb"}{"ccc"}=1;  # auto creates each of the intermediate levels
                               # of the hash (aka: dict or associated array)

Subir