Why do __setattr__ and __delattr__ raise an AttributeError in this case?

Question

In Python, what is the rationale for which object.__setattr__ and type.__setattr__ raise an AttributeError during attribute update if the type has an attribute which is a data descriptor without a __set__ method? Likewise, what is the rationale for which object.__delattr__ and type.__delattr__ raise an AttributeError during attribute deletion if the type has an attribute which is a data descriptor without a __delete__ method?

I am asking this because I have noticed that object.__getattribute__ and type.__getattribute__ do not raise an AttributeError during attribute lookup if the type has an attribute which is a data descriptor without a __get__ method.

Here is a simple program illustrating the differences between attribute lookup by object.__getattribute__ on the one hand (AttributeError is not raised), and attribute update by object.__setattr__ and attribute deletion by object.__delattr__ on the other hand (AttributeError is raised):

class DataDescriptor1:  # missing __get__
    def __set__(self, instance, value): pass
    def __delete__(self, instance): pass

class DataDescriptor2:  # missing __set__
    def __get__(self, instance, owner=None): pass
    def __delete__(self, instance): pass

class DataDescriptor3:  # missing __delete__
    def __get__(self, instance, owner=None): pass
    def __set__(self, instance, value): pass

class A:
    x = DataDescriptor1()
    y = DataDescriptor2()
    z = DataDescriptor3()

a = A()
vars(a).update({'x': 'foo', 'y': 'bar', 'z': 'baz'})

a.x
# actual: returns 'foo'
# expected: returns 'foo'

a.y = 'qux'
# actual: raises AttributeError: __set__
# expected: vars(a)['y'] == 'qux'

del a.z
# actual: raises AttributeError: __delete__
# expected: 'z' not in vars(a)

Here is another simple program illustrating the differences between attribute lookup by type.__getattribute__ on the one hand (AttributeError is not raised), and attribute update by type.__setattr__ and attribute deletion by type.__delattr__ on the other hand (AttributeError is raised):

class DataDescriptor1:  # missing __get__
    def __set__(self, instance, value): pass
    def __delete__(self, instance): pass

class DataDescriptor2:  # missing __set__
    def __get__(self, instance, owner=None): pass
    def __delete__(self, instance): pass

class DataDescriptor3:  # missing __delete__
    def __get__(self, instance, owner=None): pass
    def __set__(self, instance, value): pass

class M(type):
    x = DataDescriptor1()
    y = DataDescriptor2()
    z = DataDescriptor3()

class A(metaclass=M):
    x = 'foo'
    y = 'bar'
    z = 'baz'

A.x
# actual: returns 'foo'
# expected: returns 'foo'

A.y = 'qux'
# actual: raises AttributeError: __set__
# expected: vars(A)['y'] == 'qux'

del A.z
# actual: raises AttributeError: __delete__
# expected: 'z' not in vars(A)

I would expect the instance dictionary to be mutated instead of getting an AttributeError for attribute update and attribute deletion. Attribute lookup returns a value from the instance dictionary, so I am wondering why attribute update and attribute deletion do not use the instance dictionary as well (like they would do if the type did not have an attribute which is a data descriptor).

Voluminous

Answer 1

I think it's just a consequence of the C-level design that no one really thought or cared much about.

At C level, __set__ and __delete__ correspond to the same C-level slot, tp_descr_set, and deletion is specified by passing a null value to set. (This is similar to the design used for __setattr__ and __delattr__, which also correspond to a single slot that also gets passed NULL for deletion.)

If you implement either __set__ or __delete__, the C-level slot gets set to a wrapper function that looks for __set__ or __delete__ and calls it:

static int
slot_tp_descr_set(PyObject *self, PyObject *target, PyObject *value)
{
    PyObject* stack[3];
    PyObject *res;
    _Py_IDENTIFIER(__delete__);
    _Py_IDENTIFIER(__set__);

    stack[0] = self;
    stack[1] = target;
    if (value == NULL) {
        res = vectorcall_method(&PyId___delete__, stack, 2);
    }
    else {
        stack[2] = value;
        res = vectorcall_method(&PyId___set__, stack, 3);
    }
    if (res == NULL)
        return -1;
    Py_DECREF(res);
    return 0;
}

The slot has no way to say "oops, didn't find the method, go back to normal handling", and it doesn't try. It also doesn't try to emulate the normal handling - that would be error-prone, since "normal handling" is type-dependent, and it can't know what to emulate for all types. If the slot wrapper doesn't find the method, it just raises an exception.

This effect wouldn't happen if __set__ and __delete__ had gotten two slots, but someone would have had to care while they were designing the API, and I doubt anyone did.

Schleswigholstein

As you seem to suggest, the fundamental problem seems to be that the function slot_tp_descr_set returns -1 both when the looked up __set__ or __delete__ method is absent and when the looked up __set__ or __delete__ method is present but its call raises an exception (frequently AttributeError). So the calling function _PyObject_GenericSetAttrWithDict has no way to make the distinction and fall back on the instance in the former case. – Vevina

I don't think this is necessarily undesigned. Note that right now, code written in Python can always count on disallowed attribute operations to raise AttributeError. Tried to get or delete an attribute that doesn't exist? AttributeError. Tried to set an attribute which isn't allowed to exist on that type of object? AttributeError. And if the object's __{get,set,del}attr__ or __getattribute__ or a descriptor's __{get,set,delete}__ want to deliberately say "this doesn't exist" or "this can't be set", the way to say that is AttributeError. – Bearskin

So then if someone wants to create an attribute on an object which doesn't allow some of those operations, the most natural and in many ways the most useful way to signal that to the user of that object is the established Python-native way of saying "you can't do that on this attribute" (raise AttributeError). And for the developer of descriptors that behave that way, it's very ergonomic to not have to write boilerplate def __{get,set,delete}__(self, ...): raise AttributeError methods every time they don't want an operation to work. – Bearskin

So users of an object can always trust that operations on attributes, anything with a foo.bar syntax shape, in pretty much all cases, all follow the protocol of raising AttributeError to say "that's not allowed/supported/valid". It's basically a standard API that permeates the language, a contract between all objects and all attribute operations on those objects. Things don't raise other errors from foo.bar-shaped operations unless something went wrong internally or they have a good reason to let you distinguish error cases (but that's usually best done by subclassing AttributeError). – Bearskin

@mtraceur: But if you want a read-only descriptor, you have to manually implement __set__ and/or __delete__, and if you want a write-only descriptor, you have to manually implement __get__, even though these methods will only raise AttributeError. Implementing a subset of methods doesn't automatically default the rest of the methods to "forbidden". The descriptor slot overlap only applies to one specific pair of methods, __set__ and __delete__. – Schleswigholstein

@Schleswigholstein Yes, I was wrongly thinking of the user experience of using @property. I retract the 2nd half of my 2nd comment. But the bigger point is that raising AttributeError when you only define one of the __{set,delete}__ pair is the only choice here that matches established meaning/usage of AttributeError, the only choice that matches other things that will happen if you omit some of the methods (if I only implement __get__, then trying to delete the attribute before setting it will raise an AttributeError), and the only choice that doesn't require different error handling. – Bearskin

To be clear, I'm not saying it was definitely carefully intentionally chosen - I'm just saying that I don't see enough reason to rule it out, because I see plausible good reasons to raise AttributeError by default in cases like these. (Oh, also, set and delete are both mutating, so a descriptor defines only one of them, why are asker's expectations a more sensible default than erroring? If set does something custom to save state who-knows-where, how do you know if default delete behavior is sensible and not silent misbehavior? Ditto for default set behavior if delete is customized?) – Bearskin

@mtraceur: Sure, there are reasons to think of AttributeError as a sensible default here, but if that were the motivation, the design would look different, and the AttributeError would probably be documented somewhere, which it doesn't seem to be. – Schleswigholstein

I've gone through the original descriptor PEP and the python-dev mailing list archives, and it doesn't look like anyone thought about deletion at all in the original design - deletion wasn't mentioned or supported at all in the original design, and __delete__ was only put in after someone sent in a bug report about NULL handling (and it took a second bug report before they went to the __delete__ name - the first bug fix called it __del__, a name that was already taken). – Schleswigholstein

Thanks for doing that digging. That's good to know. For what it's worth, I often don't document or mention design decisions if I think no one wants to hear them enough to justify the conciseness loss, and I don't think the design would necessarily look different if someone thought of it (I could see myself coming up with the design that we have - not necessarily saying it's the best design, but that I could find it satisfactory enough to stop there, and hard to think of a strict improvement). But I'm now leaning your way on this: evidence mounting that no one deliberately thought about it. – Bearskin