What is the order of evaluation in python when using pop(), list[-1] and +=?

Question

a = [1, 2, 3]
a[-1] += a.pop()

This results in [1, 6].

a = [1, 2, 3]
a[0] += a.pop()

This results in [4, 2]. What order of evaluation gives these two results?

Alban

@tobias_k Take the first example. a.pop() returns 3 and changes a to [1,2]. Now evaluate a[-1] = a[-1] + 3 and you get 5, not 6. – Alban

@felipa My guess it that Python first translates the a[-1] += a.pop() to a[-1] = a[-1] + a.pop(). That is why you get the 6. a[-1] is evaluated before the a.pop(). If you change it to a[-1] = a.pop() + a[-1] you get 5 – Chiastolite

All tough a good question I would try to avoid situations like these. – Writhen

Despite the multiple comments on this page appearing to suggest the contrary, please note that in general a += b is not the same as a = a + b and is not guaranteed to be translated as such by Python. Different methods are invoked. Whether or it boils down too the same thing depends on the types of the operands. – Morgun

@ajcr That's very interesting! Could you expand on that please, maybe in an answer? – Alban

I'm not sure if it's in scope for this answer, but you can find a detailed explanation here. Basically + calls the __add__ method whereas += tries to call the __iadd__ method - these might have the same effect (e.g. integers) or they might be slightly different (e.g. lists). – Morgun

@Chris_Rands: I tried to phrase my comment carefully to say that the two were not equivalent "in general" and that it depends on the objects: I agree it's reasonable enough here. (But on a another note, I'm not sure that the fact that + and += operators are equivalent for integers is key to answering this question: the general result would be the same if different objects were involved. I think the evaluation order is the main point.) – Morgun

@ajcr Deciding what is on the RHS in order to consider the evaluation oder seems to be subtle though, as you have pointed out. – Alban

b = [[5], [3]]; a = b[1]; b[-1] += b.pop(); print (a) is even more confusing! – Alban

PSA: Never write code that relies on these kinds of details. Even if it works, it's not good code. If you isolate side effects on their own line (This amounts approximately to one state modification per line.), reading and modifying your code will be a lot easier. In this case, you should do a = [1, 2, 3]; temp = a.pop(); a[-1] = 2 * temp or a = [1, 2, 3]; temp = a.pop(); a[-1] += temp, depending on what you meant to do. This makes your intended order of evaluation explicit, and it's easier to get right. – Sonjasonnet

Answer 1

RHS first and then LHS. And at any side, the evaluation order is left to right.

a[-1] += a.pop() is same as, a[-1] = a[-1] + a.pop()

a = [1,2,3]
a[-1] = a[-1] + a.pop() # a = [1, 6]

See how the behavior changes when we change the order of the operations at RHS,

a = [1,2,3]
a[-1] = a.pop() + a[-1] # a = [1, 5]

Whitneywhitson

"at the RHS, it's left to right" Fun fact: While the operators are of course evaluated w.r.t. operator precedence, the actual expressions apparently are not, e.g. in f() + g() * h(), the functions are evaluated in order f, then g, then h. For instance, a.pop() + a.pop() * a.pop() with a = [3, 2, 1] yields 7 (1 + 2 * 3) – Borisborja

@Borisborja well, the * operator is evaluated before the + operator. It may be helpful to consider the function call to be another operator with higher precedence than *. – Sammer

@Sammer That part is clear; what stumped me was that in (a+(b+(c+(d+(...)))), a is evaluated first, then b, etc. But I guess that's just my "human" perspective, as I would visually parse the expression and determine the innermost part to evaluate that first, so I don't have to keep as much "in memory" (literally). A computer, of course, can just evaluate the expression left to right and push the intermediate results onto a stack. Still though this worth pointing out, as others might have the same wrong intuition. – Borisborja

@tobias_k: Lots of languages do this, with the notable exception of C and C++. – Epicurean

@Rob a[-1] is evaluated before the value is popped off, giving a value of 3. Then the pop occurs also giving a value of 3. Thus the math is 3+3. This is because on the RHS we work left-to-right evaluating each part by itself then dealing with the math. – Engross

@Borisborja The general rule in many languages is that parentheses are used to override operator precedence, but not order of evaluation. So it just affects how different operations are grouped. So the innermost + will be done first, but that doesn't mean d has to be evaluated first. – Plyler

Answer 2

The key insight is that a[-1] += a.pop() is syntactic sugar for a[-1] = a[-1] + a.pop(). This holds true because += is being applied to an immutable object (an int here) rather than a mutable object (relevant question here).

The right hand side (RHS) is evaluated first. On the RHS: equivalent syntax is a[-1] + a.pop(). First, a[-1] gets the last value 3. Second, a.pop() returns 3. 3 + 3 is 6.

On the Left hand side (LHS), a is now [1,2] due to the in-place mutation already applied by list.pop() and so the value of a[-1] is changed from 2 to 6.

Ento

a[-1] += a.pop() is shorthand for a[-1] = a[-1] + a.pop() is only true because a is a list of ints, I have now learned. It's worth mentioning. – Alban

@felipa Ok, yes I've added an edit, note if a was a list of strings or tuples, it would also behave the same as the integer case (it's only different for mutable objects) – Ento

Answer 3

Let's have a look at the output of dis.dis for a[-1] += a.pop()¹⁾:

3    15 LOAD_FAST            0 (a)                             # a,
     18 LOAD_CONST           5 (-1)                            # a, -1
     21 DUP_TOP_TWO                                            # a, -1, a, -1
     22 BINARY_SUBSCR                                          # a, -1, 3
     23 LOAD_FAST            0 (a)                             # a, -1, 3, a
     26 LOAD_ATTR            0 (pop)                           # a, -1, 3, a.pop
     29 CALL_FUNCTION        0 (0 positional, 0 keyword pair)  # a, -1, 3, 3
     32 INPLACE_ADD                                            # a, -1, 6
     33 ROT_THREE                                              # 6, a, -1
     34 STORE_SUBSCR                                           # (empty)

The meaning of the different instructions is listed here.

First, LOAD_FAST and LOAD_CONST load a and -1 onto the stack, and DUP_TOP_TWO duplicates the two, before BINARY_SUBSCR gets the subscript value, resulting in a, -1, 3 on the stack. It then loads a again, and LOAD_ATTR loads the pop function, which is called with no arguments by CALL_FUNCTION. The stack is now a, -1, 3, 3, and INPLACE_ADD adds the top two values. Finally, ROT_THREE rotates the stack to 6, a, -1 to match the order expected by STORE_SUBSCR and the value is stored.

So, in short, the current value of a[-1] is evaluated before calling a.pop() and the result of the addition is then stored back to the new a[-1], irrespective of its current value.

---

¹⁾ This is the disassembly for Python 3, slightly compressed to better fit on the page, with an added column showing the stack after # ...; for Python 2 it looks a bit different, but similar.

Borisborja

Cool thing with dis.dis, didn't know this one! – Nonet

Answer 4

Using a thin wrapper around a list with debugging print-statements can be used to show the order of evaluation in your cases:

class Test(object):
    def __init__(self, lst):
        self.lst = lst

    def __getitem__(self, item):
        print('in getitem', self.lst, item)
        return self.lst[item]

    def __setitem__(self, item, value):
        print('in setitem', self.lst, item, value)
        self.lst[item] = value

    def pop(self):
        item = self.lst.pop()
        print('in pop, returning', item)
        return item

When I now run your example:

>>> a = Test([1, 2, 3])
>>> a[-1] += a.pop()
in getitem [1, 2, 3] -1
in pop, returning 3
in setitem [1, 2] -1 6

So it starts by getting the last item, which is 3, then pops the last item which is also 3, adds them and overwrites the last item of your list with 6. So the final list will be [1, 6].

And in your second case:

>>> a = Test([1, 2, 3])
>>> a[0] += a.pop()
in getitem [1, 2, 3] 0
in pop, returning 3
in setitem [1, 2] 0 4

This now takes the first item (1) adds it to the popped value (3) and overwrites the first item with the sum: [4, 2].

---

The general order of evaluation is already explained by @Fallen and @tobias_k. This answer just supplements the general principle mentioned there.

Nehru

Answer 5

For you specific example

a[-1] += a.pop() #is the same as 
a[-1] = a[-1] + a.pop() # a[-1] = 3 + 3

Order:

1. evaluate a[-1] after =
2. pop(), decreasing the length of a
3. addition
4. assignment

The thing is, that a[-1] becomes the value of a[1] (was a[2]) after the pop(), but this happens before the assignment.

a[0] = a[0] + a.pop() 

Works as expected

1. evaluate a[0] after =
2. pop()
3. addition
4. assignment

This example shows, why you shouldn't manipulate a list while working on it (commonly said for loops). Always work on copys in this case.

Nonet