What does the operator += return in Python

Question

Original Question

While I was trying to answer another person's question on stackoverflow about the difference between = and += in Python, I encountered the following problem:

class Foo:
    def __init__(self, value, name="default"):
        self.value = value
        self.name = name
        
    def __add__(self, that):
        return Foo(self.value + that.value)
    
    def __iadd__(self, that):
        self.value = self.value + that.value
        return self
    
    def __str__(self):
        return "name: {}, value: {:d}".format(self.name, self.value)
    
a = Foo(1, 'alice')
b = Foo(2, 'bob')
print(a+=b)

The last print call was not successful and gave me this:

File "<ipython-input-8-0faa82ba9e4a>", line 3
    print(a+=b)
            ^
SyntaxError: invalid syntax

I don't know why this isn't working. Maybe it has something to do with the keyword argument passing mechanism? I just can't find any resource on this topic, since the overloaded __iadd__ method already returns a Foo object.

************** update ******************

If I change the __iadd__ method like this (just remove the return statement):

...
    def __iadd__(self, that):
        print("__iadd__ was called")
        self.value = self.value + that.value

a = Foo(1, 'alice')
b = Foo(2, 'bob')
a += b
print(a)  # Outputs: None

So, the final return statement in __iadd__ is indeed required. But it does not function as I thought (I come from a C/C++ background, so this behavior is a bit strange for me)

************************* 2nd Update ********************************

I almost forget that = in Python makes up a statement instead of an expression. The return statement in __iadd__ and my experience with other languages gives me an illusion that += could be used as an expression.

As stated in the Python documentation, __add__ is used to construct a new object. __iadd__ is designed for inplace modifications. But it all depends on the implementation. Although __iadd__ returns a object, this object is somehow "intercepted" by the language and reassigned to the left-hand operator, and, the final effect is, __iadd__ remains a statement, not an expression. Please correct me if I'm wrong. Also, I didn't find any resource confirming that += is a statement.

Summary

1. A plain code, say a = 1 is an assignment statement. It's not allowed to be used as a function argument. However, keyword argument passing is not restricted by that: print('Hello world', end='') still works.
2. • x = x + y is equivalent to x = x.__add__(y),
   • x += y is equivalent to x = x.__iadd__(y), check the doc for more details.
3. An example:

class Foo:
    def __init__(self, value, name="default"):
        self.value = value
        self.name = name

    def __add__(self, that):
        return Foo(self.value + that.value)

    def __iadd__(self, that):
        self.value = self.value + that.value
        return self

    def __str__(self):
        return "name: {}, value: {:d}".format(self.name, self.value)

a = Foo(1, 'alice')
b = Foo(2, 'bob')
c = a + b # objects a and b are unchanged
a += b    # object a is changed
print(c) # name: default, value: 3
print(a) # name: alice, value: 3

Prelate

+= returns an object that is reassigned to the left operand. But assignment statements are not expressions, so you can't use them as a value. – Prussiate

Assignment expressions don't return anything – Coerce

@IainShelvington because they aren't expressions! Assignment expressions, with the "walrus operator" do return the value being assigned – Lipid

a += b is a statement, not an expression. You can not print it like you can not print(a = a + b) – Shouldst

Thanks for the fast reply. For me it's weird to make += return something that cannot be used as a expression like in C/C++. The implementations of __add__ and __iadd__ are almost identical but why one of them can be used as a expression, the other one can't? – Prelate

+ would be useless if it didn't give you an expression. The purpose of += is to update the left operand. It's part of Python's design that actions to do one thing don't incidentally do something else as well. – Prussiate

visit this link you might find answer: #4841936 – Burwell

@ShihaoXu Make sure to read docs.python.org/3/reference/datamodel.html#object.__iadd__ – Shouldst

@ShihaoXu Also, as @Lipid has already alluded to, in Python >= 3.8 you can use the assignment expression operator to modify a in-place and print the result: a = 1 ; b = 2 ; print(a := a + b) ; print(a) gives the output 3 ; 3 – Shouldst

@ShihaoXu as per your update, it needs to be able to return something (for the assignment to occur) otherwise how would the operator work with immutable objects – Leiker

@Shouldst Thanks for the link to python's doc. Actually I've already read that specific part. As stated in the doc, __iadd__ does return something. And it doesn't mention that augmented assignments are statements instead of expressions. – Prelate

@SamMason I think the assignment is not necessary if I don't use += "inplace". The value in self is changed anyway. So I guess Python uses the returned object for the "final" assignment, since it will return None by default. Anyway, in other languages like C++, it won't destroy your object completely if you forget to return itself in a normal method. – Prelate

@ShihaoXu well, the += statement doesn't return anything, because it isn't an expression. The hook __iadd__ does, because that hook is used by Python to control the behavior of += – Lipid

It is important to understand, the "magic method" hooks like __iadd__ and __add__ are not one-to-one equivalent with += and +. They are hooks – Lipid

"__iadd__ remains a statement, not an expression" no. __iadd__ is a method, that method is used as a hook when the statement ` a += b` is executed. It doesn't even have to exist for that statement to evaluate just fine (e.g., it will fall back to the __add__ hook if not implemented) – Lipid

@Lipid Thanks! Your answer is very helpful. Hooks are just hooks. I think my problem stems from the (fake) one-to-one correspondence :-p between the hooks and operators. – Prelate

@Lipid Thanks for reminding the := operator. I'm forced to stay with Python3.5 due to some CUDA stuff. As you can see I didn't even use a f-string. But it's good to know that Python is finally equipped with an assignment expression. – Prelate

Answer 1

The semantics of Python might be confusing matters. As an illustrative example, try working through the following example:

class Foo:
    def __init__(self, value):
        self.value = value

    def __iadd__(self, other):
        self.value = self.value + other.value
        return 'did iadd'

a = Foo(1)
c = a
c += Foo(2)

print((c, a.value))

should result in ('did iadd', 3) being printed

As per the docs:

x += y is equivalent to x = x.__iadd__(y)

The contract for __iadd__ is that you should implement it in such a way that the semantics of x = x + y evaluates to the "same" as x += y. The "same" depends on the details of the problem, e.g. for immutable objects (such as integers or strings) this will necessitate allocating a new object and updating the reference to point to, e.g.:

a = 500  # move out of the "interned" number space
c = a
a += 1
print(id(a), id(c))

should give you two different numbers, which in CPython will be the addresses of the two int objects, both distinct and immutable. For other data types, it can be useful to allow mutable/inplace updates to occur, e.g. for efficiency or other reasons. The semantics of __iadd__ allow for the class implementation to choose a default.

Each language has its own peculiarities, hence I'd suggest not trying to compare it too literally to C++. Especially as C++ has a lot of historical baggage it's being forced to carry along.

Leiker

Indeed, x += y is the same as x = x.__iadd__(y). The most "popular" answer suggesting that x += y is the same as x = x + y is confusing, because x += y calls __iadd__ while x = x + y calls __add__, both followed by an assignment. I think this point is very clear to me now. Thanks! – Prelate

Answer 2

Assignments/in-place operations can't be considered as expressions that can be passed on, and that's by design.

Among other things, Guido wanted to avoid the infamous C typo

if (a=b)  // user actually wanted to test a==b

But python 3.8 provides a new assignment method known as assignment expression that allows to pass the assigned value as a parameter (which is very handy in list comprehensions).

You still cannot do in-place add but you can do add then assign:

>> a=2
>> b=3
>> print(a:=a+b)
5

Paraglider

Answer 3

This is a syntax error caused by using an assignment statement as an expression. Here is another SO post which explains the difference between statements and expressions.

It is not a bug with the implementation of the operator overload.

Example of how the syntax results in an error in another context (using int instead of Foo):

a = 2
b = 3

# STILL AN ERROR!
print(a += b) # Error while using the standard int += operator

How to fix the syntax issue:

a = 2
b = 3

a += b      # Separate the assignment statement...
print(a)    #     ...and give the print function an expression

Hangout

It has everything to do with the operator. The operator is what makes that line a statement (ie "unprintable") – Shouldst

@Shouldst thanks for pointing out that my wording was ambiguous. I updated my post to clarify that the bug does not originate from the implementation, but from how it is being used. – Hangout

Answer 4

The += operator is part of a set of multiple Augmented Assignment Operators. You can think of these as a shortcut for Assignment Statements where the variable being assigned is also in the value being assigned to the variable. So, i += 1 is identical to i = i + 1.

Thus, the issue that you are encountering is caused because only values (integer, float, strings, etc.) or expressions can be passed to the print() function. a += b is not a value (it is a statement) and it is not an expression (it does not evaluate to a single value), so you cannot pass it to print().

To solve this:

a = Foo(1, 'alice')
b = Foo(2, 'bob')
a += b
print(a)

Edwardoedwards