How can I make a deepcopy of a function in Python?

Question

I would like to make a deepcopy of a function in Python. The copy module is not helpful, according to the documentation, which says:

This module does not copy types like module, method, stack trace, stack frame, file, socket, window, array, or any similar types. It does “copy” functions and classes (shallow and deeply), by returning the original object unchanged; this is compatible with the way these are treated by the pickle module.

My goal is to have two functions with the same implementation but with different docstrings.

def A():
    """A"""
    pass

B = make_a_deepcopy_of(A)
B.__doc__ = """B"""

So how can this be done?

Olive

Do you want them to have the same __name__? – Tart

Eh, I'm not too concerned about that, as I could easily change it once I have a copy. – Olive

Answer 1

The FunctionType constructor is used to make a deep copy of a function.

import types
def copy_func(f, name=None):
    return types.FunctionType(f.func_code, f.func_globals, name or f.func_name,
        f.func_defaults, f.func_closure)

def A():
    """A"""
    pass
B = copy_func(A, "B")
B.__doc__ = """B"""

Dilatation

@GlennMaynard : this can’t work for built‑in functions https://mcmap.net/q/67639/-how-can-i-make-a-deepcopy-of-a-function-in-python/2284570 – Dispersoid

Only works in python 2. See Aaron Hall's post below for an answer that's py 2/3 compatible. – Salesclerk

Answer 2

My goal is to have two functions with the same implementation but with different docstrings.

Most users will do this, say the original function is in old_module.py:

def implementation(arg1, arg2): 
    """this is a killer function"""

and in new_module.py

from old_module import implementation as _implementation

def implementation(arg1, arg2):
    """a different docstring"""
    return _implementation(arg1, arg2)

This is the most straightforward way to reuse functionality. It is easy to read and understand the intent.

Nevertheless, perhaps you have a good reason for your main question:

How can I make a deepcopy of a function in Python?

To keep this compatible with Python 2 and 3, I recommend using the function's special __dunder__ attributes. For example:

import types

def copy_func(f, name=None):
    '''
    return a function with same code, globals, defaults, closure, and 
    name (or provide a new name)
    '''
    fn = types.FunctionType(f.__code__, f.__globals__, name or f.__name__,
        f.__defaults__, f.__closure__)
    # in case f was given attrs (note this dict is a shallow copy):
    fn.__dict__.update(f.__dict__) 
    return fn

And here's an example usage:

def main():
    from logging import getLogger as _getLogger # pyflakes:ignore, must copy
    getLogger = copy_func(_getLogger)
    getLogger.__doc__ += '\n    This function is from the Std Lib logging module.\n    '
    assert getLogger.__doc__ is not _getLogger.__doc__
    assert getLogger.__doc__ != _getLogger.__doc__

A commenter says:

This can’t work for built‑in functions

Well I wouldn't do this for a built-in function. I have very little reason to do this for functions written in pure Python, and my suspicion is that if you are doing this, you're probably doing something very wrong (though I could be wrong here).

If you want a function that does what a builtin function does, and reuses the implementation, like a copy would, then you should wrap the function with another function, e.g.:

_sum = sum
def sum(iterable, start=0):
    """sum function that works like the regular sum function, but noisy"""
    print('calling the sum function')
    return _sum(iterable, start)
    

Boren

This can’t work for built‑in functions https://mcmap.net/q/67639/-how-can-i-make-a-deepcopy-of-a-function-in-python/2284570 – Dispersoid

py 2/3 compatibility for the win! – Salesclerk

This solution of a duplicate problem is very similar, but also takes into account a few further subtleties such as __kwdefaults__. – Foxhound

Maybe this technique isn't well motivated for docstrings, but I'm glad you wrote the tutorial because I really needed it! I needed a copy of a suite of functions, which call each other, to be copied to a new (dynamic) module and call each others' copies, rather than the originals. This meant replacing their __globals__ to point to the new module. The reason for all of this was to Numba-compile all of the copied functions without affecting the original functions, since they still need to be accessed in pure object mode. So thanks! – Pamulapan

Answer 3

from functools import partial

def a():
    """Returns 1"""
    return 1

b = partial(a)
b.__doc__ = """Returns 1, OR DOES IT!"""

print help(a)
print help(b)

Wrap it as a partial?

Yb

So the only reason I think this is not as "nice" is that from an IPython terminal, one cannot look at the source of b, via "b??". Strictly a personal preference. – Olive

God damn it! try the bpython console – Yb

@JakobBowyer : using this solution you don’t get a copy of a, but you wrap a in a functools.partial object. – Dispersoid

b = lambda *args, **kwargs: a(*args, **kwargs) works as well, but a bit more verbose – Felly

Answer 4

def A():
    """A"""
    pass

def B():
    """B"""
    return A()

Absorptance

It works, but is there something nicer...which doesn't increase the function call overhead? I literally just want a copy. – Olive

I guess, even if there would be a way, it won't be 'nicer' – Absorptance

I agree it won't be nicer, but since I am calling this function in for loops nested a few levels down, I went with the factory method. – Olive

Answer 5

The others answers do not allow for serialization with pickle. Here a code that I am using to clone a function and allow for serialization for python3:

import pickle
import dill
import types

def foo():
    print ('a')


oldCode=foo.__code__

name='IAmFooCopied'

newCode= types.CodeType(
        oldCode.co_argcount,             #   integer
        oldCode.co_kwonlyargcount,       #   integer
        oldCode.co_nlocals,              #   integer
        oldCode.co_stacksize,            #   integer
        oldCode.co_flags,                #   integer
        oldCode.co_code,                 #   bytes
        oldCode.co_consts,               #   tuple
        oldCode.co_names,                #   tuple
        oldCode.co_varnames,             #   tuple
        oldCode.co_filename,             #   string
        name,                  #   string
        oldCode.co_firstlineno,          #   integer
        oldCode.co_lnotab,               #   bytes
        oldCode.co_freevars,             #   tuple
        oldCode.co_cellvars              #   tuple
        )

IAmFooCopied=types.FunctionType(newCode, foo.__globals__, name,foo.__defaults__ , foo.__closure__)
IAmFooCopied.__qualname__= name
print ( 'printing foo and the copy', IAmFooCopied, foo )
print ( 'dill output: ', dill.dumps(IAmFooCopied ))
print ( 'pickle Output: ', pickle.dumps (IAmFooCopied) )

Output:

printing foo and the copy <function IAmFooCopied at 0x7f8a6a8159d8> <function foo at 0x7f8a6b5f5268>
dill output:  b'\x80\x03cdill._dill\n_create_function\nq\x00(cdill._dill\n_load_type\nq\x01X\x08\x00\x00\x00CodeTypeq\x02\x85q\x03Rq\x04(K\x00K\x00K\x00K\x02KCC\x0ct\x00d\x01\x83\x01\x01\x00d\x00S\x00q\x05NX\x01\x00\x00\x00aq\x06\x86q\x07X\x05\x00\x00\x00printq\x08\x85q\t)X\x10\x00\x00\x00testCloneFunc.pyq\nX\x0c\x00\x00\x00IAmFooCopiedq\x0bK\x05C\x02\x00\x01q\x0c))tq\rRq\x0ec__builtin__\n__main__\nh\x0bNN}q\x0ftq\x10Rq\x11.'
pickle Output:  b'\x80\x03c__main__\nIAmFooCopied\nq\x00.'

You may encounter problem with the qualname attribute if you try this snippet with class methods (I think pickle should fail to find your function). I never tried it, however it should be easily fixable. Just check the doc about qualname

Janellejanene

somehow people overlooked this piece – Manthei

Brilliant. This is undeniably the optimally efficient solution to function copying both here and elsewhere. Sadly, it lacks upvotes. Sadly, it's also incomplete. Why? Because it fails to copy the __kwdefaults__, __module__, __qualname__, __doc__, and __annotations__ attributes from the old to new function. Fortunately, all of those attributes can be trivially copied by just calling functools.update_wrapper(IAmFooCopied, foo, functools.WRAPPER_ASSIGNMENTS + ('__kwdefaults__',)). Yup. – Cupcake

Addendum: the new CodeType.replace() method introduced in Python 3.8 is strongly preferable to reconstructing a new CodeType object from scratch. Why? Because future-proofing. The CodeType constructor frequently changes between Python versions, which then breaks everything. – Cupcake

Answer 6

It's quite easy to do using lambda and rest parameters:

def my_copy(f): 
        # Create a lambda that mimics f
        g = lambda *args: f(*args)
        # Add any properties of f
        t = list(filter(lambda prop: not ("__" in prop),dir(f)))
        i = 0
        while i < len(t):
            setattr(g,t[i],getattr(f,t[i]))
            i += 1
        return g
        
# Test
def sqr(x): return x*x
sqr.foo = 500

sqr_copy = my_copy(sqr)
print(sqr_copy(5)) # -> 25
print(sqr_copy(6)) # -> 36
print(sqr_copy.foo) # -> 500
print(sqr_copy == sqr) # -> False

Try it online!

Chromous

Answer 7

put it in a function:

def makefunc( docstring ):
    def f():
        pass
    f.__doc__ = docstring
    return f

f = makefunc('I am f')
g = makefunc("I am f too")

Racemic

Alright, I guess I am going to cave in and accept this. I was hoping there was something which didn't rely or wrapping or a construction, but I suppose it ultimately comes down to that at some level. – Olive

@Olive did you not look at my example? It shows basically the syntax you want. – Yb

Answer 8

Adjusted for python3

import types
def copy_func(f, name=None):
    return types.FunctionType(f.__code__, f.__globals__, name or f.__name__,
        f.__defaults__, f.__closure__)
def func1(x):
  return 2*x
func2=copy_func(func1)
print(func2(7))

Lean

Answer 9

I've implemented a general-purpose function copy in haggis, a library which I wrote and maintain (available with pip but probably not conda). haggis.objects.copy_func makes a copy on which you can not only reassign the __doc__ attribute, but also modify the module and __globals__ attributes effectively.

from haggis.objects import copy_func

def a(*args, **kwargs):
    """A docstring"""

a2 = copy_func(a)
a2.__doc__ = """Another docstring"""

>>> a == a2
False
>>> a.__code__ == a2.__code__
True
>>> a.__doc__
'A docstring'
>>> a2.__doc__
'Another docstring'

Portage