Why does mutating a list in a tuple raise an exception but mutate it anyway? [duplicate]

Question

I am not sure I quite understand what's happening in the below mini snippet (on Py v3.6.7). It would be great if someone can explain to me as to how can we mutate the list successfully even though there's an error thrown by Python.

I know that we can mutate a list and update it, but what’s with the error? Like I was under the impression that if there's an error, then the x should remain the same.

x = ([1, 2], )
x[0] += [3,4] # ------ (1)

The Traceback thrown at line (1) is

> TypeError: 'tuple' object doesn't support item assignment.. 

I understand what the error means but I am unable to get the context of it.

But now if I try to print the value of my variable x, Python says it's,

print(x) # returns ([1, 2, 3, 4])

As far as I can understand, the exception has happened after Python allowed the mutation of the list to happen and then hopefully it tried re-assigning it back. It blew there I think as Tuples are immutable.

Can someone explain what's happening under the hood?

Edit - 1 Error From ipython console as an image;

Stealing

@Aaron So that means we should not use += ? Is there some kind of trade-off for in-place ops wrt list? – Stealing

x[0].extend([3,4]) does work as expected however... – Rebatement

Simple in-place assignment x += y is always welcome. When slicing / index / attribute is used, considering mutability is needed. – Okajima

I appreciate all the Answers! They all are useful and unique but can only accept one :(; Thanks folk's! – Stealing

Answer 1

My gut feeling is that the line x[0] += [3, 4] first modifies the list itself so [1, 2] becomes [1, 2, 3, 4], then it tries to adjust the content of the tuple which throws a TypeError, but the tuple always points towards the same list so its content (in terms of pointers) is not modified while the object pointed at is modified.

We can verify it that way:

a_list = [1, 2, 3]
a_tuple = (a_list,)
print(a_tuple)
>>> ([1, 2, 3],)

a_list.append(4)
print(a_tuple)
>>> ([1, 2, 3, 4], )

This does not throw an error and does modify it in place, despite being stored in a "immutable" tuple.

Lama

x[0] = x[0] + [3,4] doesn't work so this seems to be related to += – Chiaki

@Okajima makes an excellent point. The remainder of my comment here seems wrong: +=t is shorthand for + on the two operands followed by assignment back to the left operand. The interpreter performs the + concatenation on the mutable array stored in x[0] and does not raise an error until the assignment step attempts to mutate the immutable tuple. – Roane

I trust your gut; So we should avoid using +=? And prefer extend/append always? – Stealing

Because __iadd__ mutates the list, __add__ creates a new one. – Gaultiero

@Stealing It really depends on what you are doing. Out of context it's a bit difficult to give a definitive answer. Maybe x could be a list in your case, maybe extend/append are prefered if you need to keep x as a tuple. Personally I use += a lot for lists because I like how easy it is to read, so I really can't tell you to always extend instead. – Lama

Answer 2

There are a few things happening here.

+= is not always + and then =.

+= and + can have different implementations if required.

Take a look at this example.

In [13]: class Foo: 
    ...:     def __init__(self, x=0): 
    ...:         self.x = x 
    ...:     def __add__(self, other): 
    ...:         print('+ operator used') 
    ...:         return Foo(self.x + other.x) 
    ...:     def __iadd__(self, other): 
    ...:         print('+= operator used') 
    ...:         self.x += other.x 
    ...:         return self 
    ...:     def __repr__(self): 
    ...:         return f'Foo(x={self.x})' 
    ...:                                                                        

In [14]: f1 = Foo(10)                                                           

In [15]: f2 = Foo(20)                                                           

In [16]: f3 = f1 + f2                                                           
+ operator used

In [17]: f3                                                                     
Out[17]: Foo(x=30)

In [18]: f1                                                                     
Out[18]: Foo(x=10)

In [19]: f2                                                                     
Out[19]: Foo(x=20)

In [20]: f1 += f2                                                               
+= operator used

In [21]: f1                                                                     
Out[21]: Foo(x=30)

Similarly, the list class has separate implementations for + and +=.

Using += actually does an extend operation in the background.

In [24]: l = [1, 2, 3, 4]                                                       

In [25]: l                                                                      
Out[25]: [1, 2, 3, 4]

In [26]: id(l)                                                                  
Out[26]: 140009508733504

In [27]: l += [5, 6, 7]                                                         

In [28]: l                                                                      
Out[28]: [1, 2, 3, 4, 5, 6, 7]

In [29]: id(l)                                                                  
Out[29]: 140009508733504

Using + creates a new list.

In [31]: l                                                                      
Out[31]: [1, 2, 3]

In [32]: id(l)                                                                  
Out[32]: 140009508718080

In [33]: l = l + [4, 5, 6]                                                      

In [34]: l                                                                      
Out[34]: [1, 2, 3, 4, 5, 6]

In [35]: id(l)                                                                  
Out[35]: 140009506500096

---

Let's come to your question now.

In [36]: t = ([1, 2], [3, 4])                                                   

In [37]: t[0] += [10, 20]                                                       
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-37-5d9a81f4e947> in <module>
----> 1 t[0] += [10, 20]

TypeError: 'tuple' object does not support item assignment

In [38]: t                                                                      
Out[38]: ([1, 2, 10, 20], [3, 4])

The + operator gets executed first here, which means the list gets updated (extended). This is allowed as the reference to the list (value stored in the tuple) doesn't change, so this is fine.

The = then tries to update the reference inside the tuple which isn't allowed since tuples are immutable.

But the actual list was mutated by the +.

Python fails to update the reference to the list inside the tuple but since it would have been updated to the same reference, we, as users don't see the change.

So, the + gets executed and the = fails to execute. + mutates the already referenced list inside the tuple so we see the mutation in the list.

Rhoea

Answer 3

Existing answers are correct, but I think the documentation can actually shed some extra light on this:

From in-place operators documentation:

the statement x += y is equivalent to x = operator.iadd(x, y)

so when we write

x[0] += [3, 4]

It's equivalent to

x[0] = operator.iadd(x[0], [3, 4])

iadd is implemented using extend, in the case of the list, so we see that this operation is actually doing 2 things:

• extend the list
• re-assign to index 0

As indicated later in the documentation:

note that when an in-place method is called, the computation and assignment are performed in two separate steps.

The first operation is not a problem

The second operation is impossible, since x is a tuple.

But why re-assigning?

This can seem puzzling in this case, and one can wonder why the +=operator is equivalent to x = operator.iadd(x, y), rather than simply operator.iadd(x, y).

This wouldn't work for immutable types, like int and str. So while iadd is implemented as return x.extend(y) for lists, it is implemented as return x + y for ints.

Again from the documentation:

For immutable targets such as strings, numbers, and tuples, the updated value is computed, but not assigned back to the input variable

Letterpress