How to stop attributes from being pickled in Python [duplicate]

Question

I am using gnosis.xml.pickle to convert an object of my own class to xml. The object is initialized so that:

self.logger = MyLogger()

But when I do dump the object to a string I get an exception stating that the pickler encountered an unpickleable type (thread.lock).

Is there a way to 'tag' the logger attribute so that pickler will know not to try and pickle that attribute?

Nowadays

Answer 1

You can define two methods, __getstate__ and __setstate__, to your class to override the default pickling behavior.

http://docs.python.org/library/pickle.html#object.__getstate__

__getstate__ should return a dict of attributes that you want to pickle.

def __getstate__(self):
    d = dict(self.__dict__)
    del d['logger']
    return d

__setstate__ should setup your object with the provided dict.

def __setstate__(self, d):
    self.__dict__.update(d) # I *think* this is a safe way to do it

Note that __init__ won't be called when unpickling so you'll have to create your logger in __setstate__

Stencil

thanks. please note that i am using the gnosis pickle lib, not sure if it would work the same? – Nowadays

I googled for the gnosis source and saw calls to getstate and setstate, so I think it will work. – Stencil

I found that the __setstate__ as written there wasn't needed since I didn't need the logger in other threads. – Rustcolored

Answer 2

This might be a better solution since it will allow an object created via copy.deepcopy to still have a self.logger:

def __getstate__(self):
    d = self.__dict__.copy()
    if 'logger' in d:
        d['logger'] = d['logger'].name
    return d

def __setstate__(self, d):
    if 'logger' in d:
        d['logger'] = logging.getLogger(d['logger'])
    self.__dict__.update(d)

Mattingly

Answer 3

Your class can implement the special method __getstate__ to return exactly what parts of an instance it wants to be pickled.

There are several possible variants on that (though __getstate__ and its optional companion method __setstate__ are most general) -- see the online Python doc page for pickle, to which I already pointed above because it's the one documenting __getstate__.

Illyricum

thanks. ideally i would like to only have to specify what i do NOT want to pickle. – Nowadays

@Ries, loop over all members (using inspect.getmembers, the instance's __dict__, whatever) and remove those you want to remove. There is no __donotsavetheseones_ special method, you have to implement it as __getstate__. – Illyricum