Overcoming Python's limitations regarding instance methods

Question

It seems that Python has some limitations regarding instance methods.

1. Instance methods can't be copied.
2. Instance methods can't be pickled.

This is problematic for me, because I work on a very object-oriented project in which I reference instance methods, and there's use of both deepcopying and pickling. The pickling thing is done mostly by the multiprocessing mechanism.

What would be a good way to solve this? I did some ugly workaround to the copying issue, but I'm looking for a nicer solution to both problems.

Does anyone have any suggestions?

Update:

My use case: I have a tiny event system. Each event has an .action attribute that points to a function it's supposed to trigger, and sometimes that function is an instance method of some object.

Meghanmeghann

Usually, we provide the code in all locations in which we're working and merely move a representation of the state around. Representation State Transfer is more common and simpler. What's wrong with that? – Effusive

S.Lott, you clearly have more experience with this issue, but please explain it to me in a more dumbed-down way, what exactly are you suggesting, how does it relate, what is "Representation State Transfer", etc. – Meghanmeghann

You're saying "I want to pickle instance methods", but you're not saying why. It would help us to know what problem you're trying to solve, since there may be a more Pythonic way to do it. It sounds a bit like you're thinking in Ruby... – Fic

Jonathan: I have a tiny event system. Each event has an .action attribute that points to a function it's supposed to trigger, and sometimes that function is an instance method of some object. – Meghanmeghann

@cool-RR: Please do not comment on your own question. It's your question. You own it. Please update the question with additional information. – Effusive

Answer 1

You might be able to do this using copy_reg.pickle. In Python 2.6:

import copy_reg
import types

def reduce_method(m):
    return (getattr, (m.__self__, m.__func__.__name__))

copy_reg.pickle(types.MethodType, reduce_method)

This does not store the code of the method, just its name; but that will work correctly in the common case.

This makes both pickling and copying work!

Epigone

Good idea, I think it might be the cleanest solution. Do you have any idea for the other problem, copying? Also, did anyone already implement a copy_reg solution that I can use? – Meghanmeghann

Answer 2

REST - Representation State Transfer. Just send state, not methods.

To transfer an object X from A to B, we do this.

1. A encode the state of X in some handy, easy-to-parse notation. JSON is popular.

2. A sends the JSON text to B.

3. B decodes the state of X form JSON notation, reconstructing X.

B must have the class definitions for X's class for this to work. B must have all functions and other class definitions on which X's class depends. In short, both A and B have all the definitions. Only a representation of the object's state gets moved around.

See any article on REST.

http://en.wikipedia.org/wiki/Representational_State_Transfer

http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm

Effusive

Can you please explain how this methodology relates to my problem with the event system? – Meghanmeghann

This is a "you're doing it wrong" answer. This view is that pickle is maybe a little too convenient in Python; it's really better to exchange Plain Old Data (as opposed to object graphs) because it results in simpler systems that are saner to reason about. There's something in that, but if it's helpful to you at all it'll be helpful in a long-term way. – Epigone

Answer 3

pickle the instance and then access the method after unpickling it. Pickling a method of an instance doesn't make sense because it relies on the instance. If it doesn't, then write it as an independent function.

import pickle

class A:
     def f(self):
         print 'hi'

x = A()
f = open('tmp', 'w')
r = pickle.dump(x, f)
f.close()
f = open('tmp', 'r')
pickled_x = pickle.load(f)
pickled_x.f()

Pecos

A() isn't an instance method. – Epigone

You pickled the object, not the instance method. Try pickling an instance method directly. – Meghanmeghann

pickle the instance and then access the method after unpickling it. Pickling a method of an instance doesn't make sense because it relies on the instance. If it doesn't, then write it as an independent function. – Pecos