Understanding Python's call-by-object style of passing function arguments [duplicate]

Question

I am not sure I understand the concept of Python's call by object style of passing function arguments (explained here http://effbot.org/zone/call-by-object.htm). There don't seem to be enough examples to clarify this concept well (or my google-fu is probably weak! :D)

I wrote this little contrived Python program to try to understand this concept

def foo( itnumber, ittuple,  itlist, itdict   ):
    itnumber +=1 
    print id(itnumber) , itnumber 

    print id(ittuple)  , ittuple

    itlist.append(3.4)
    print id(itlist)   , itlist

    itdict['mary']  = 2.3
    print id(itdict),    itdict



# Initialize a number, a tuple, a list and a dictionary
tnumber = 1
print id( tnumber ), tnumber 

ttuple  = (1, 2, 3)
print id( ttuple ) , ttuple

tlist   = [1, 2, 3]
print id( tlist ) , tlist

tdict = tel = {'jack': 4098, 'sape': 4139}
print '-------'
# Invoke a function and test it
foo(tnumber, ttuple, tlist , tdict)

print '-------'
#Test behaviour after the function call is over
print id(tnumber) , tnumber 
print id(ttuple)  , ttuple
print id(tlist)   , tlist
print id(tdict),  tdict

The output of the program is

146739376 1
3075201660 (1, 2, 3)
3075103916 [1, 2, 3]
3075193004 {'sape': 4139, 'jack': 4098}

---------

146739364 2
3075201660 (1, 2, 3)
3075103916 [1, 2, 3, 3.4]
3075193004 {'sape': 4139, 'jack': 4098, 'mary': 2.3}

---------

146739376 1
3075201660 (1, 2, 3)
3075103916 [1, 2, 3, 3.4]
3075193004 {'sape': 4139, 'jack': 4098, 'mary': 2.3}

As you can see , except for the integer that was passed, the object id's (which as I understand refers to memeory location) remain unchanged.

So in the case of the integer, it was (effectively) passed by value and the other data structure were (effectively) passed by reference. I tried changing the list , the number and the dictionary to just test if the data-structures were changed in place. The number was not bu the list and the dictionary were.

I use the word effectively above, since the 'call-by-object' style of argument passing seems to behave both ways depending on the data-structure passed in the above code

For more complicated data structures, (say numpy arrays etc), is there any quick rule of thumb to recognize which arguments will be passed by reference and which ones passed by value?

Obligor

Since you seem to understand C, Python's 'pass by object' is similar to passing pointers by value, with some values pointed to (like tuples and integers) immutable. – Castor

What @Castor said. But it's better to avoid the by-value/by-reference paradigm altogether and just think about things and names. – Riancho

Answer 1

The key difference is that in C-style language, a variable is a box in memory in which you put stuff. In Python, a variable is a name.

Python is neither call-by-reference nor call-by-value. It's something much more sensible! (In fact, I learned Python before I learned the more common languages, so call-by-value and call-by-reference seem very strange to me.)

In Python, there are things and there are names. Lists, integers, strings, and custom objects are all things. x, y, and z are names. Writing

x = []

means "construct a new thing [] and give it the name x". Writing

x = []
foo = lambda x: x.append(None)
foo(x)

means "construct a new thing [] with name x, construct a new function (which is another thing) with name foo, and call foo on the thing with name x". Now foo just appends None to whatever it received, so this reduces to "append None to the the empty list". Writing

x = 0
def foo(x):
    x += 1
foo(x)

means "construct a new thing 0 with name x, construct a new function foo, and call foo on x". Inside foo, the assignment just says "rename x to 1 plus what it used to be", but that doesn't change the thing 0.

Riancho

Thank you for the nice answer. Just to clarify your point about things and names I tested the following on ipython >>> id(3.14) gives 145793796 >>> x=3.14 >>> id(x) gives 145793796 >>> >>> >>> id(2) gives 145756324 >>> x=2 >>> id(x) gives 145756324 >>> So indeed, as you said, x is a name which is being rebound to different objects. Thank you. – Obligor

I like this answer except for your last example. The problem is that += is supposed to work differently for mutable and immutable objects. A mutable object very well could have visible changes outside of foo. – Snowshoe

Important point about +=: a += b may actually end up calling a.__iadd__(b), which is just the same as calling any other method of a, it doesn't do any reassignment. If a doesn't have an __iadd__ function then a += b reduces to a = a + b, which ends up reducing to a = a.__add__(b) or a = b.__radd__(a) – Matazzoni

Answer 2

Others have already posted good answers. One more thing that I think will help:

 x = expr

evaluates expr and binds x to the result. On the other hand:

 x.operate()

does something to x and hence can change it (resulting in the same underlying object having a different value).

The funny cases come in with things like:

 x += expr

which translate into either x = x + expr (rebinding) or x.__iadd__(expr) (modifying), sometimes in very peculiar ways:

>>> x = 1
>>> x += 2
>>> x
3

(so x was rebound, since integers are immutable)

>>> x = ([1], 2)
>>> x
([1], 2)
>>> x[0] += [3]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
>>> x
([1, 3], 2)

Here x[0], which is itself mutable, was mutated in-place; but then Python also attempted to mutate x itself (as with x.__iadd__), which errored-out because tuples are immutable. But by then x[0] was already mutated!

Performing

I should probably admit, I first saw this on comp.lang.python and have no idea who showed it. It sure is peculiar though! – Performing

Could you share more info as for why Python does not prevent this assignment even though it throws an error? I can imagine some reasons related to this behaviour but I'd like to know the implementation details. – Outspan

@Arn: it's just a matter of operation sequence: the += operator calls x[0].__iadd__ first. Then, the semantics of += say that Python should re-assign the object on the left (see docs.python.org/3/reference/…). The problem could be prevented if the "evaluate left hand side" operation were performed with an additional flag, "with intent to assign": Python could then catch the error before calling __iadd__. But it isn't, and doesn't. – Performing

Answer 3

Numbers, strings, and tuples in Python are immutable; using augmented assignment will rebind the name.

Your other types are merely mutated, and remain the same object.

Embolectomy