Emulating pass-by-value behaviour in python

Question

I would like to emulate the pass-by-value behaviour in python. In other words, I would like to make absolutely sure that the function I write do not modify user supplied data.

One possible way is to use deep copy:

from copy import deepcopy
def f(data):
    data = deepcopy(data)
    #do stuff

is there more efficient or more pythonic way to achieve this goal, making as few assumptions as possible about the object being passed (such as .clone() method)

Edit

I'm aware that technically everything in python is passed by value. I was interested in emulating the behaviour, i.e. making sure I don't mess with the data that was passed to the function. I guess the most general way is to clone the object in question either with its own clone mechanism or with deepcopy.

Kimbra

You are already doing it right. – Contrabandist

Just a terminology note: Python already uses pass-by-value. Many values just happen to be references, so you're passing references by value. If Python used pass-by-reference then def f(x): x = 5 would actually change the value of the expression the caller was passing for the caller, but it doesn't. – Vary

@L.Gonsalves Neither "pass-by-value" or "pass-by-reference" describe Python's semantics. See for example mail.python.org/pipermail/python-list/2007-July/621112.html – Rafa

@dF: the page you linked to is... "confused". Python uses pass-by-value. It is also true that all variables in Python are references to values, but that doesn't change the fact that the parameter passing is by value. The page itself acknowledges that Java passes by value, and non-primitive types in Java are stored as references. If we removed primitive types from Java would it no longer be pass-by-value? No. Likewise, the fact that Python variables always store references is orthogonal to the fact that its parameter passing is by value. – Vary

@L.Gonsalves I see what you mean, and I agree the page I linked isn't very clear. I guess my point is that if you're coming from a language such as C, "pass by value" implies that a copy of the data is made, which Python never does. – Rafa

@ Laurence, this is a common misconception. Python is actually all pass by reference. Variable assignment is a pointer binding, it does not actually change the data structure that the variable points to. Things like strings and integers which seem to be the exception are not really exceptions. They are immutable, and therefore any operations on them that yield a different string/int actually create new ones if they didn't exist already. – Albuminuria

Answer 1

There is no pythonic way of doing this.

Python provides very few facilities for enforcing things such as private or read-only data. The pythonic philosophy is that "we're all consenting adults": in this case this means that "the function shouldn't change the data" is part of the spec but not enforced in the code.

---

If you want to make a copy of the data, the closest you can get is your solution. But copy.deepcopy, besides being inefficient, also has caveats such as:

Because deep copy copies everything it may copy too much, e.g., administrative data structures that should be shared even between copies.

[...]

This module does not copy types like module, method, stack trace, stack frame, file, socket, window, array, or any similar types.

So i'd only recommend it if you know that you're dealing with built-in Python types or your own objects (where you can customize copying behavior by defining the __copy__ / __deepcopy__ special methods, there's no need to define your own clone() method).

Rafa

Implementing your own __deepcopy__ also typically speeds up a call to deepcopy by many factors, if you must use deepcopy. – Paratuberculosis

Answer 2

You can make a decorator and put the cloning behaviour in that.

>>> def passbyval(func):
def new(*args):
    cargs = [deepcopy(arg) for arg in args]
    return func(*cargs)
return new

>>> @passbyval
def myfunc(a):
    print a

>>> myfunc(20)
20

This is not the most robust way, and doesn't handle key-value arguments or class methods (lack of self argument), but you get the picture.

Note that the following statements are equal:

@somedecorator
def func1(): pass
# ... same as ...
def func2(): pass
func2 = somedecorator(func2)

You could even have the decorator take some kind of function that does the cloning and thus allowing the user of the decorator to decide the cloning strategy. In that case the decorator is probably best implemented as a class with __call__ overridden.

Engaged

I think a decorator is a really elegant approach to solving this problem. I'm sure you could come up with something that could cope with keyword args too without much trouble. – Bradford

Yep, it's more or less a matter of giving the new method a **kwargs argument too, and copying them. – Engaged

Answer 3

There are only a couple of builtin typs that work as references, like list, for example.

So, for me the pythonic way for doing a pass-by-value, for list, in this example, would be:

list1 = [0,1,2,3,4]
list2 = list1[:]

list1[:] creates a new instance of the list1, and you can assign it to a new variable.

Maybe you could write a function that could receive one argument, then check its type, and according that resulta, perform a builtin operation that could return a new instance of the argument passed.

As I said earlier, there are only a few builtin types, that their behavior is like references, lists in this example.

Any way... hope it helps.

Sagunto

everything in python is passed and assigned by value. all values in Python are references. there are no types that work differently than any others – Kampmeier

Your way naively did what I wanted it to do. Thanks. – Dewyeyed

Answer 4

I can't figure out any other pythonic option. But personally I'd prefer the more OO way.

class TheData(object):
  def clone(self):  """return the cloned"""

def f(data):
  #do stuff

def caller():
  d = TheData()
  f(d.clone())

Observer

Thank you. However, clone implies a customized and maybe more efficient way to do deep copy. I personally saw libraries where the cloned objects were not as deep as one expected :) . Not to mention classes with no clone method at all. So, basically deepcopy is more portable. – Kimbra

(see my clarification to the question) – Kimbra

deepcopy is simply another way to clone the data. The real point in my code is that "inside the caller" is indeed the better place to decide on how to pass the argument, not inside the function – Observer

Also deepcopy is absolutely not "more efficient". When the decision to clone the data is within the caller where you know exactly the kind of data you are passing, then you can use the most efficient method to do that; (like list[:], dict(dic)).. If you are not really sure what data will be passed, then keeping the decision for efficient-cloning-method to the data object itself will always yield better performance. – Observer

Do I understand you right, that the clone()/deepcopy() ought to reside with the data structure. Then you use a caller method that gets a function f passed? (In that case there's the f parameter missing in caller(f).) Wouldn't you need to know the function's signature first? Can you make it generically multivariate? Or will you implement a new data structure for every function? Or define every function as a member function to every data object? – Leonoraleonore

I mean -- define clone() as a member function to every data object that you want to pass-by-value. – Observer

Answer 5

usually when passing data to an external API, you can assure the integrity of your data by passing it as an immutable object, for example wrap your data into a tuple. This cannot be modified, if that is what you tried to prevent by your code.

Satterfield

Answer 6

Further to user695800's answer, pass by value for lists possible with the [:] operator

def listCopy(l):
    l[1] = 5
    for i in l:
        print i

called with

In [12]: list1 = [1,2,3,4]

In [13]: listCopy(list1[:])
1
5
3
4

list1
Out[14]: [1, 2, 3, 4]

Prissy

Answer 7

Though i'm sure there's no really pythonic way to do this, i expect the pickle module will give you copies of everything you have any business treating as a value.

import pickle

def f(data):
    data = pickle.loads(pickle.dumps((data)))
    #do stuff

Indentation

Answer 8

Many people use the standard library copy. I prefer to defining __copy__ or __deepcopy__ in my classes. The methods in copy may have some problems.

1. The shallow copy will keep references to the objects in the original instead of creating new one.
2. The deepcopy will run recursively, sometimes may cause dead-loop. If without enough attention, memory may explode.

To avoid these out-of-control behaviors, define your own shallow/deep copy methods through overwriting __copy__ and __deepcopy__. And Alex's answer give a good example.

Lamella