Using Python descriptors with slots

Question

I want to be able use python descriptors in a class which has the slots optimization:

class C(object):    
    __slots__ = ['a']
    a = MyDescriptor('a')
    def __init__(self, val):
        self.a = val

The problem I have is how to implement the descriptor class in order to be able to store values in the class instance which invokes the descriptor object. The usual solution would look like the one below but will not work since "dict" is no longer defined when "slots" is invoked in the C class:

class MyDescriptor(object):
    __slots__ = ['name']    
    def __init__(self, name_):
        self.name = name_
    def __get__(self, instance, owner):
        if self.name not in instance.__dict__:
            raise AttributeError, self.name
        return instance.__dict__[self.name]     
    def __set__(self, instance, value):
        instance.__dict__[self.name] = value

Pummel

Answer 1

Don't declare the same name as a slot and as an instance method. Use different names, and access the slot as an attribute, not via __dict__.

class MyDescriptor(object):
    __slots__ = ['name']
    def __init__(self, name_):
        self.name = name_
    def __get__(self, instance, owner):
        return getattr(instance, self.name)
    def __set__(self, instance, value):
        setattr(instance, self.name, value)

class C(object):
    __slots__ = ['_a']
    a = MyDescriptor('_a')
    def __init__(self, val):
        self.a = val

foo = C(1)
print foo.a
foo.a = 2
print foo.a

Froe

After executing that, I can see that C.__dict__ is created. Isn't that something we are trying to avoid? Anyway, looking at the documentation __slots__ is already using descriptor interface for its own purpose, so this could be really difficult. – Program

Using Python 3, C has only a mappinyproxy and not a dict. More importantly though, instances of C don't have a __dict__ – Bulkhead

Answer 2

Though of dubious value, the following trick will work, if it is ok to put the first __slots__ in a subclass.

class A( object ):
    __slots__ = ( 'a', )

class B( A ):
    __slots__ = ()

    @property
    def a( self ):
        try:
            return A.a.__get__( self )
        except AttributeError:
            return 'no a set'

    @a.setter
    def a( self, val ):
        A.a.__set__( self, val )

(You can use your own descriptor rather than property.) With these definitions:

>>> b = B()
>>> b.a
'no a set'
>>> b.a = 'foo'
>>> b.a
'foo'

As far as I understand, __slots__ is implemented with its own descriptor, so another descriptor after __slots__ in the same class would just overwrite. If you want to elaborate this technique, you could search for a candidate descriptor in self.__class__.__mro__ (or starting with instance in your own __get__).

Postscript

Ok ... well if you really want to use one class, you can use the following adaptation:

class C( object ):
    __slots__ = ( 'c', )

class MyDescriptor( object ):

    def __init__( self, slots_descriptor ):
        self.slots_descriptor = slots_descriptor

    def __get__( self, inst, owner = None ):
        try:
            return self.slots_descriptor.__get__( inst, owner )
        except AttributeError:
            return 'no c'

    def __set__( self, inst, val ):
        self.slots_descriptor.__set__( inst, val )

C.c = MyDescriptor( C.c )

If you insist on inscrutability, you can make the assignment in a metaclass or a class decorator.

Sharice

Answer 3

The @Glenn Maynard's answer is the good one.

But I would like to point at a problem in the OP's question (I can't add a comment to his question since I havn't enough reputation yet):

The following test is raising an error when the instance hasn't a __dict__ variable:

        if self.name not in instance.__dict__:

So, here is an a generic solution that tries to acces to the __dict__ variable first (which is the default anyway) and, if it fails, use getattr and setattr:

class WorksWithDictAndSlotsDescriptor:

    def __init__(self, attr_name):
        self.attr_name = attr_name

    def __get__(self, instance, owner):
        try:
            return instance.__dict__[self.attr_name]
        except AttributeError:
            return getattr(instance, self.attr_name)

    def __set__(self, instance, value):
        try:
            instance.__dict__[self.attr_name] = value
        except AttributeError:
            setattr(instance, self.attr_name, value)

(Works only if the attr_name is not the same as the real instance variable's name, or you will have a RecursionError as pointed to in the accepted answer)

(Won't work as expected if there is both __slots__ AND __dict__)

Hope this helps.

Titanite