Python descriptors with old-style classes

Question

I tried to google something about it. Why do non-data descriptors work with old-style classes?

Docs say that they should not:
"Note that descriptors are only invoked for new style objects or classes (ones that subclass object() or type()).".

class Descriptor(object):
    def __init__(self):
        self.x = 1

    def __get__(self, obj, cls=None):
        return self.x


class A:
    x = Descriptor()

a = A()
a.x

>>> 1

Thanks.

Konopka

Because 'Descriptor' is a new-style class. – Heterocercal

Answer 1

You are right to question the documentation. I've tried looking through CPython sources to find an explanation, but be warned: I'm no expert.

From my understanding, attribute lookup and descriptor __get__ invocation occurs in instance_getattr2 (chosen extracts):

v = class_lookup(inst->in_class, name, &klass);
if (v != NULL) {
    f = TP_DESCR_GET(v->ob_type);
    if (f != NULL) {
        PyObject *w = f(v, (PyObject *)inst, (PyObject *)(inst->in_class));
    }
}

So either I am missing something, or nothing in the implementation requires a new-style object (which contradicts the documentation).

For the record, I tried recompiling Python to restrict descriptor invocation to new style classes objects, but it actually brought up a gigantic mess. I learned in the process that class methods themselves are implemented as descriptors: this is the mechanism used to return bound or unbound method objects depending on the usage. For example:

>>> class A:
...     def foo():
...         pass
...
>>> A.foo.__get__(None, A)
<unbound method A.foo>
>>> A.foo.__get__(A(), A)
<bound method A.foo of <__main__.A instance at 0x000000000229CC48>>

As a result, it seems that preventing descriptor invocation for attributes of old-style objects or classes would also prevent method calls on them, at least with CPython implementation.

Once again, I'm no expert and this is the first time I dive into Python implementation, so I could very well be wrong. I've filed an issue to try to clarify this.

Livelihood

Thanks for the investigation, it is really helpful. – Konopka

I edited my answer after a debugging session in CPython: I had the wrong functions involved, but the result stays the same. – Livelihood

In Python all functions (including methods) are non-data descriptors. So they should act like you say. – Konopka

But I still do have one question. Why __get__ does not work if a Descriptor is an old-style class? I think that if an attribute or method of old-style (or new-style) class is subinstance of object (all functions are), the interpreter trying to execute __get__. – Konopka

Note that in python3 there unbound methods are gone. They are simply function attributes of classes. @Konopka I believe the limitation is not on the class using the descriptor(A in your example), but on the class that implements the descriptor(Descriptor). If Descriptor is an old-style class, then __get/set/del__ are not considered like special-methods, hence they aren't called. When the Descriptor class inherits object then those methods are considered special. – Kingfisher

The documentation is backwards/misleading: "... descriptors are only invoked on new style ..." is trying to say "descriptor protocol methods (__get__, __set__, and __del__) are only invoked on new style ...". So the descriptor need to be new-style, not the class using the descriptor. – Libya