How to use `__slots__` with initialization of attributes?

Question

I read through the main answers on usage of slots and it has given me an idea of how and where to use __slots__ .

Now, I am porting a code from Python 2 to Python 3 which is similar to as following:

class B(object):
    __slots__ = ('_fields')
    _fields = set()

But this is giving error Python 3 while working fine on Python 2:

ValueError: '_fields' in __slots__ conflicts with class variable.

I change the code to

class B(object):
    __slots__ = ('_fields')
    def __init__(self):
        _fields = set()

and it works fine. My query is, is it even the correct change ?

As i got from original code, I guess it is saying that don't use __dict__ for saving memory or faster access or whatever reason but at the same time is also trying to specify the type of attribute _field as set(). Can the change above be the right way to say it or it can have some side effects.

---

Further Experiments: I experimented further with following variants (on Python 3):

import pdb

class A(object):
    a = set()

'''
class B(object):
    __slots__ = ('a')
    a = set()
'''

class C(object):
    __slots__ = ('a')
    def __init__(self):
        a = set()

class D(object):
    def __init__(self):
        __slots__ = ('a')
        a = set()

if __name__ == '__main__':
    #pdb.set_trace()
    x = A(); print(dir(x))
    #y = B()
    z = C(); print(dir(z))
    z1 = D(); print(dir(z1))

and it gave following output.

['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'a']


['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__slots__', '__str__', '__subclasshook__', 'a']


['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__']

We can see that only C object shows correct footprint i.e. no __dict__ and only __slots__ . Isn't it what ideally we would want ? Any explanation on __weakref__ would also be helpful.

Also on Python 2, both B and C object show same footprint. Based on that should C be the right way to put it as it is compiling on both Python 2 and 3 as well.

Mephistopheles

Answer 1

But this is giving error Python 3 while working fine on Python 2:

ValueError: '_fields' in __slots__ conflicts with class variable.

While you didn't get an error in Python2 at class creation/compile time like in Py3k, if you try to actually set the value of _fields, you get AttributeError: 'C' object attribute '_fields' is read-only:

>>> class C(object):
...   __slots__ = ('_fields')
...   _fields = set()
...
>>>
>>> c = C()
>>> c._fields
set([])
>>> c._fields = 'foo'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'C' object attribute '_fields' is read-only
>>>

Also, see the fourth note in the slots documentation:

__slots__ are implemented at the class level by creating descriptors (Implementing Descriptors) for each variable name. As a result, class attributes cannot be used to set default values for instance variables defined by __slots__; otherwise, the class attribute would overwrite the descriptor assignment.

---

Wrt your modification:

I change the code to

class B(object):
    __slots__ = ('_fields')
    def __init__(self):
        _fields = set()

The modified class B has a _fields inside __init__(), not self._fields so it's just a local variable in init, and not accessible anywhere else in the class. Change that to:

 class B(object):
    __slots__ = ('_fields')
    def __init__(self):
        self._fields = set()

To correctly initialise _fields, do this:

 class B(object):
     __slots__ = ('_fields')
     def __init__(self, _fields=None):
         if not _fields:
             self._fields = set()

---

Wrt further experimentation:

In class D, __slots__ is a variable only inside __init()__. It's not the (special) class variable D.__slots__; or even the instance variable self.__slots__. So it has __dict__.

Class A has none, so also has __dict__.

Class C has __slots__ correctly.

Dovetailed

Thanks. But did we want to say the following in __init__ instead self._fields = _fields if not None else set() – Mephistopheles

if not _fields: checks _fields in boolean context. So if _fields is None then that expression evaluates to True and the line self._fields = set() gets executed. It could also be done as if _fields is None: self._fields = set(). Btw, in the example in your comment, not None is always True. You could put it as self._fields = _fields if _fields is not None else set() or the more Pythonic self._fields = _fields if _fields else set(). Caveat: an empty set() evaluates to False so use an appropriate check... – Dovetailed

Caveat: an empty set() evaluates to False so use an another check if set() is okay but None is not. Also, don't intialise function or methods with mutable types as default argument values; like set, dict, list, etc. Another example here. – Dovetailed

Answer 2

The easiest way I personally found to solve this issue:

class B(object):
    __slots__ = ('_fields')
    _fields: set()

    # Overridden in the derived classes
    def __init__(self, _fields=None):
    # your code

Skantze

Answer 3

(Asssume Python3)

A class attribute does not have to be mentioned in __slots__. In other words, a class attribute can be defined even if the class is derived from object and its name does not appear in the class' __slots__.

The correct way to achieve this in your case is:

class B(object):
    __slots__ = ('x', 'y', 'z') # e.g. restrict instance attributes (if wanted)
    _fields = set()             # define the class attribute independently.

Butyraldehyde

Answer 4

I just had a (stupid?) idea and I'm really not sure if this is "valid" Python, but it seems to work (quickly tested in Python 3.7.7):

class Slotted:

    __slots__ = {}

    def __new__(cls, *args, **kwargs):
        inst = super().__new__(cls)
        for key, value in inst.__slots__.items():
            setattr(inst, key, value)
        return inst

class Magic(Slotted):

    __slots__ = {
        "foo": True,
        "bar": 17
    }

Note that I'm (wrongly?) using a __slots__ dictionary here!

magic = Magic()

print(f"magic.foo = {magic.foo}")
print(f"magic.bar = {magic.bar}")

magic.foo = True
magic.bar = 17

Does anyone know if there are disadvantages of doing this?

Edlyn

Sorry for the duplication but I've just created a new question from my own answer to get more feedback on the matter. – Edlyn

Slots defined as dict are meant to contain docstrings for the attribute not default values! – Kucera