Why does deepcopy fail with "KeyError: '__deepcopy__'" when copying custom object?

Question

I have a class that converts a dictionary to an object like this

class Dict2obj(dict):
    __getattr__= dict.__getitem__

    def __init__(self, d):
        self.update(**dict((k, self.parse(v))
                           for k, v in d.iteritems()))

   @classmethod
   def parse(cls, v):
    if isinstance(v, dict):
        return cls(v)
    elif isinstance(v, list):
        return [cls.parse(i) for i in v]
    else:
        return v

When I try to make a deep copy of the object I get this error

import copy
my_object  = Dict2obj(json_data)
copy_object = copy.deepcopy(my_object)

File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/copy.py", line 172, in deepcopy
copier = getattr(x, "__deepcopy__", None)
KeyError: '__deepcopy__'

But if I override the __getattr__ function in the Dict2obj class I was able to do deep copy operation. See example below

class Dict2obj(dict):

    __getattr__= dict.__getitem__

    def __init__(self, d):
        self.update(**dict((k, self.parse(v))
                           for k, v in d.iteritems()))

    def __getattr__(self, key):
        if key in self:
            return self[key]
        raise AttributeError

    @classmethod
    def parse(cls, v):
        if isinstance(v, dict):
            return cls(v)
        elif isinstance(v, list):
            return [cls.parse(i) for i in v]
        else:
            return v

Why do I need to override __getattr__ method in order to do a deepcopy of objects returned by this class?

Civilian

Answer 1

The issue occurs for your first class, because copy.deepcopy tries to call getattr(x, "__deepcopy__", None) . The significance of the third argument is that, if the attribute does not exist for the object, it returns the third argument.

This is given in the documentation for getattr() -

getattr(object, name[, default])

Return the value of the named attribute of object. name must be a string. If the string is the name of one of the object’s attributes, the result is the value of that attribute. For example, getattr(x, 'foobar') is equivalent to x.foobar. If the named attribute does not exist, default is returned if provided, otherwise AttributeError is raised.

This works as , if the underlying __getattr__ raises AttributeError and the default argument was provided for the getattr() function call the AttributeError is caught by the getattr() function and it returns the default argument, otherwise it lets the AttributeError bubble up. Example -

>>> class C:
...     def __getattr__(self,k):
...             raise AttributeError('asd')
...
>>>
>>> c = C()
>>> getattr(c,'a')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in __getattr__
AttributeError: asd
>>> print(getattr(c,'a',None))
None

But in your case, since you directly assign dict.__getitem__ to __getattr__ , if the name is not found in the dictionary, it raises a KeyError , not an AttributeError and hence it does not get handled by getattr() and your copy.deepcopy() fails.

You should handle the KeyError in your getattr and then raise AttributeError instead. Example -

class Dict2obj(dict):

    def __init__(self, d):
        self.update(**dict((k, self.parse(v))
                           for k, v in d.iteritems()))

    def __getattr__(self, name):
        try:
            return self[name]
        except KeyError:
            raise AttributeError(name)
    ...

Gotha

what if we can't change the object class code? Is there an alternative simple way to simply make a copy of the object not tethered to the original? – Klipspringer

Answer 2

Alternative to Anand's answer if you want to retain the KeyError with __getattr__ instead of an AttributeError (i.e. if you want the same error being raised as for a normal dict), you can add

__deepcopy__ = None

in your class definition.

Requirement