Why doesn't a python dict.update() return the object?

Question

I have this code:

award_dict = {
    "url": "http://facebook.com",
    "imageurl": "http://farm4.static.flickr.com/3431/3939267074_feb9eb19b1_o.png",
    "count": 1,
}

def award(name, count, points, desc_string, my_size, parent):
    if my_size > count:
        a = {
            "name": name,
            "description": desc_string % count,
            "points": points,
            "parent_award": parent,
        }
        a.update(award_dict)
        return self.add_award(a, siteAlias, alias).award

But the code felt rather cumbersome. I would have preferred to be able to write:

def award(name, count, points, desc_string, my_size, parent):
    if my_size > count:
        return self.add_award({
            "name": name,
            "description": desc_string % count,
            "points": points,
            "parent_award": parent,
        }.update(award_dict), siteAlias, alias).award

Why doesn't the update method return the original dictionary, so as to allow chaining, like how it works in JQuery? Why isn't it acceptable in python?

---

_{See How do I merge two dictionaries in a single expression in Python? for workarounds.}

Parallelism

* TL;DR newdict = dict(dict001, **dict002) – Cortney

@dreftymac, that doesn't work in comprehensions though. – Sunn

@Sunn Yes, that is indeed one valid caveat to point out. – Cortney

@dreftymac, another caveat is dict002 cannot have any integer keys (or any other non-string keys) – Inchmeal

@Sunn sure it does. newdict = {k: v for k, v in itertools.chain(dict001.items(), dict002.items())} – Veronaveronese

Answer 1

Python's mostly implementing a pragmatically tinged flavor of command-query separation: mutators return None (with pragmatically induced exceptions such as pop;-) so they can't possibly be confused with accessors (and in the same vein, assignment is not an expression, the statement-expression separation is there, and so forth).

That doesn't mean there aren't a lot of ways to merge things up when you really want, e.g., dict(a, **award_dict) makes a new dict much like the one you appear to wish .update returned -- so why not use THAT if you really feel it's important?

Edit: btw, no need, in your specific case, to create a along the way, either:

dict(name=name, description=desc % count, points=points, parent_award=parent,
     **award_dict)

creates a single dict with exactly the same semantics as your a.update(award_dict) (including, in case of conflicts, the fact that entries in award_dict override those you're giving explicitly; to get the other semantics, i.e., to have explicit entries "winning" such conflicts, pass award_dict as the sole positional arg, before the keyword ones, and bereft of the ** form -- dict(award_dict, name=name etc etc).

Sonnier

Well, that will create another dictionary after I had to make a. I wanted to create a dict, and then add a bunch of other values, and then give it to a function. – Parallelism

@Paul, and that's exactly what you're doing -- with two statements (much more readable than the nested way you wanted) which to you "felt really cumbersome". Editing my answer to show how to avoid creating a altogether, btw, – Sonnier

This isn't working for me: aa = {'aa':2}: dict(aa, {'bb':2}) outputs *** TypeError: dict expected at most 1 arguments, got 2 – Gelatinate

@DReispt Should be dict(aa, **{'bb':2}) – Regulation

@ErikAllik There are of course more efficient ways, which work for more than two dicts as well: dict(chain(d1.iteritems(), d2.iteritems())) – Regulation

Original solution is not robust. If award_dict contains keys already specified, a SyntaxError will be thrown for a repeated keyword argument. jamylak's solution dict(itertools.chain(d1.iteritems(), .. d<n>.iteritems())) not only works in the case where dictionaries have duplicate keys but also easily allows you to merge multiple dictionaries with dicts later in the chain taking precedence for the final value. – Jacquline

@Jacquline true, I guess in this case that wasn't a problem though. note that this works fine when you don't pass any specific kwargs eg. dict(d1, **d2) always works afaik – Regulation

I think this may be a CPython thing. PyPy gives me "TypeError: keywords must be strings" – Quadratic

Also, if the keys in award_dict are not string the interpreter will throw a TypeError – Evasion

Thanks so much for the answer, dict(a, **kwargs) is exactly what I was looking for. – Horbal

dict(old_dict, old_key=new_value) won't throw multiple values for keyword and return new dict. – Seadog

@AlexMartelli, it's interesting that WL (Wolfram Language) violates command-query separation by making everything functional, yet it is so much more programmer efficient than python, like 5:1 or 10:1 LOC for data workflows. – Sunn

I understand the reasoning, what I wish there was though is a way to NOT change the original dict, but create a new dict merged with the original, and return THAT. dict(a, **b) kinda does match the difference between l.reverse() and reversed(l), however **b does have problems with non-string keys :( – Ethelstan

Interestingly, dict(a, **otherdict) raises a TypeError if otherdict has non-string keys… but it doesn't in 2.7. Ah, well, purity beats pragmatism in 3.x after all… – Hower

Please avoid this solution like a plague, even when it's a good one for py2. Quoting Guido's words "I am fine with declaring dict({}, **{1:3}) illegal, since after all it is abuse of the ** mechanism." – Taxidermy

@Charmy's comment is the superior solution – Occurrence

Answer 2

Python's API, by convention, distinguishes between procedures and functions. Functions compute new values out of their parameters (including any target object); procedures modify objects and don't return anything (i.e. they return None). So procedures have side effects, functions don't. update is a procedure, hence it doesn't return a value.

The motivation for doing it that way is that otherwise, you may get undesirable side effects. Consider

bar = foo.reverse()

If reverse (which reverses the list in-place) would also return the list, users may think that reverse returns a new list which gets assigned to bar, and never notice that foo also gets modified. By making reverse return None, they immediately recognize that bar is not the result of the reversal, and will look more close what the effect of reverse is.

Stabile

Thank you. Why wouldn't reverse also give the option to not do it inplace? Performance? doing reverse(foo) feels weird. – Parallelism

Adding an option would be inappropriate: it would change the nature of the method depending on a parameter. However, methods should really have fixed return types (there are, unfortunately, cases where this rule is broken). It's easy to create a reverted copy: just make a copy (using bar=foo[:]), then revert the copy. – Hezekiah

I think the reason is explicitness. In bar = foo.reverse(), you could think that foo is not modified. To avoid confusion, you have both foo.reverse() and bar = reversed(foo). – Vinegar

What's wrong with changing the nature of a parameter based on a parameter? – Corking

@Corking it's a widely-accepted (though also widely-violated) best practice not to write such methods, because it makes the code much harder to read and understand. Especially if the value of the parameter is determined at runtime, and double especially if the method has many different parameter-selected modes, it can be very hard for a code reviewer to correctly determine what a given invocation can do. If you need to do two different things, it's better to write two different methods. – Varhol

The behavior of update is <modify but not return>. Most commenters seem to be discussing the difference between that and <modify and return>. I (and perhaps the OP?) wonder why there isn't another method that does <return but not modify>. That would be a function that computes a new value out of its parameters, and it would be useful in many contexts. – Cestar

IMO returning modified versions of an object is a really common pattern and there should be immutable functions like .updated and .reversed for this purpose. – Jevons

Answer 3

This is easy as:

(lambda d: d.update(dict2) or d)(d1)

Or, if it is important not to modify the dictionary:

(lambda d: d.update(dict2) or d)(d1.copy())

Fuentes

why? I have a class that used to return None after using return d1.update(d2). Let's consider that d2 comes from something like self.one_function(). What is the theory behind this lambda method? Thanks for the answer. – Roddie

1. If we would like to operate over d1 returned from class, and left it bound somewhere in class we have to use the first one. So this instance of dictionary will be shared inside and outside class 2. If we want just to return an updated copy let's use the second one, but, actually {**d1, **d2} is much easier to understand. 3. These examples are workarounds; I don't want to meet such code in production. Let's assume that it is a coding golf 4. lambda is needed to operate two times over one variable inside or statement; first statement in or returns None, second one - updated dict – Fuentes

Answer 4

not enough reputation for comment left on top answer

@beardc this doesn't seem to be CPython thing. PyPy gives me "TypeError: keywords must be strings"

The solution with **kwargs only works because the dictionary to be merged only has keys of type string.

i.e.

>>> dict({1:2}, **{3:4})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: keyword arguments must be strings

vs

>>> dict({1:2}, **{'3':4})
{1: 2, '3': 4}

Marcenemarcescent

Clever solution to use in one liners. – Detector

Only solution that doesn't impact the initial dictionary object. If you aim to use immutable variables that's the one ! – Larkins

Answer 5

>>> dict_merge = lambda a,b: a.update(b) or a
>>> dict_merge({'a':1, 'b':3},{'c':5})
{'a': 1, 'c': 5, 'b': 3}

Note that as well as returning the merged dict, it modifies the first parameter in-place. So dict_merge(a,b) will modify a.

Or, of course, you can do it all inline:

>>> (lambda a,b: a.update(b) or a)({'a':1, 'b':3},{'c':5})
{'a': 1, 'c': 5, 'b': 3}

Thadeus

-1 lambda should not be used like that, instead use conventional function def instead – Regulation

Don't even need a lambda, just use a.update(b) or a – Notification

Answer 6

Its not that it isn't acceptable, but rather that dicts weren't implemented that way.

If you look at Django's ORM, it makes extensive use of chaining. Its not discouraged, you could even inherit from dict and only override update to do update and return self, if you really want it.

class myDict(dict):
    def update(self, *args):
        dict.update(self, *args)
        return self

Lemures

Thank you, this could patch dict, I just wanted to know why dict() didn't allow this functionality itself (since it is as easy as you demonstrate). Does Django patch dict like this? – Parallelism

Answer 7

For those coming late to the party, I had put some timing together (Py 3.7), showing that .update() based methods look a bit (~5%) faster when inputs are preserved and noticeably (~30%) faster when just updating in-place.

As usual, all the benchmarks should be taken with a grain of salt.

def join2(dict1, dict2, inplace=False):
    result = dict1 if inplace else dict1.copy()
    result.update(dict2)
    return result


def join(*items):
    iter_items = iter(items)
    result = next(iter_items).copy()
    for item in iter_items:
        result.update(item)
    return result


def update_or(dict1, dict2):
    return dict1.update(dict2) or dict1


d1 = {i: str(i) for i in range(1000000)}
d2 = {str(i): i for i in range(1000000)}

%timeit join2(d1, d2)
# 258 ms ± 1.47 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit join(d1, d2)
# 262 ms ± 2.97 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit dict(d1, **d2)
# 267 ms ± 2.74 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit {**d1, **d2}
# 267 ms ± 1.84 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

The timings for the in-place operations are a bit trickier, so it would need to be modified along an extra copy operation (the first timing is just for reference):

%timeit dd = d1.copy()
# 44.9 ms ± 495 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

%timeit dd = d1.copy(); join2(dd, d2)
# 296 ms ± 2.05 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit dd = d1.copy(); join2(dd, d2, True)
# 234 ms ± 1.02 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit dd = d1.copy(); update_or(dd, d2)
# 235 ms ± 1.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Flunkey

Answer 8

import itertools
dict_merge = lambda *args: dict(itertools.chain(*[d.iteritems() for d in args]))

Jacquline

Answer 9

as close to your proposed solution as I could get

from collections import ChainMap

return self.add_award(ChainMap(award_dict, {
    "name" : name,
    "description" : desc_string % count,
    "points" : points,
    "parent_award" : parent,
}), siteAlias, alias).award

Anthropocentric

Answer 10

Just been trying this myself in Python 3.4 (so wasn't able to use the fancy {**dict_1, **dict_2} syntax).

I wanted to be able to have non-string keys in dictionaries as well as provide an arbitrary amount of dictionaries.

Also, I wanted to make a new dictionary so I opted to not use collections.ChainMap (kinda the reason I didn't want to use dict.update initially.

Here's what I ended up writing:

def merge_dicts(*dicts):
    all_keys  = set(k for d in dicts for k in d.keys())
    chain_map = ChainMap(*reversed(dicts))
    return {k: chain_map[k] for k in all_keys}

merge_maps({'1': 1}, {'2': 2, '3': 3}, {'1': 4, '3': 5})
# {'1': 4, '3': 5, '2': 2}

Halland

Answer 11

Merge by concatenating list of items:

d1 = {1: "one"}
d2 = {2: "two"}
dict(list(d1.items()) + list(d2.items()))
# {1: 'one', 2: 'two'}

Hampson

Answer 12

Nowadays, with Python 3.9, you can use the | operator, see:

https://docs.python.org/3.9/whatsnew/3.9.html#dictionary-merge-update-operators

Quoting from there:

>>> x = {"key1": "value1 from x", "key2": "value2 from x"}
>>> y = {"key2": "value2 from y", "key3": "value3 from y"}
>>> x | y
{'key1': 'value1 from x', 'key2': 'value2 from y', 'key3': 'value3 from y'}
>>> y | x
{'key2': 'value2 from x', 'key3': 'value3 from y', 'key1': 'value1 from x'}

Surrogate