python attribute lookup without any descriptor magic?

Question

I've started to use the python descriptor protocol more extensively in the code I've been writing. Typically, the default python lookup magic is what I want to happen, but sometimes I'm finding I want to get the descriptor object itself instead the results of its __get__ method. Wanting to know the type of the descriptor, or access state stored in the descriptor, or somesuch thing.

I wrote the code below to walk the namespaces in what I believe is the correct ordering, and return the attribute raw regardless of whether it is a descriptor or not. I'm surprised though that I can't find a built-in function or something in the standard library to do this -- I figure it has to be there and I just haven't noticed it or googled for the right search term.

Is there functionality somewhere in the python distribution that already does this (or something similar)?

Thanks!

from inspect import isdatadescriptor

def namespaces(obj):
    obj_dict = None
    if hasattr(obj, '__dict__'):
        obj_dict = object.__getattribute__(obj, '__dict__')

    obj_class = type(obj)
    return obj_dict, [t.__dict__ for t in obj_class.__mro__]

def getattr_raw(obj, name):
    # get an attribute in the same resolution order one would normally,
    # but do not call __get__ on the attribute even if it has one
    obj_dict, class_dicts = namespaces(obj)

    # look for a data descriptor in class hierarchy; it takes priority over
    # the obj's dict if it exists
    for d in class_dicts:
        if name in d and isdatadescriptor(d[name]):
            return d[name]

    # look for the attribute in the object's dictionary
    if obj_dict and name in obj_dict:
        return obj_dict[name]

    # look for the attribute anywhere in the class hierarchy
    for d in class_dicts:
        if name in d:
            return d[name]

    raise AttributeError

Edit Wed, Oct 28, 2009.

Denis's answer gave me a convention to use in my descriptor classes to get the descriptor objects themselves. But, I had an entire class hierarchy of descriptor classes, and I didn't want to begin every __get__ function with a boilerplate

def __get__(self, instance, instance_type):
    if instance is None: 
        return self
    ...

To avoid this, I made the root of the descriptor class tree inherit from the following:

def decorate_get(original_get):
    def decorated_get(self, instance, instance_type):
        if instance is None:
            return self
        return original_get(self, instance, instance_type)
    return decorated_get

class InstanceOnlyDescriptor(object):
    """All __get__ functions are automatically wrapped with a decorator which
    causes them to only be applied to instances. If __get__ is called on a 
    class, the decorator returns the descriptor itself, and the decorated
    __get__ is not called.
    """
    class __metaclass__(type):
        def __new__(cls, name, bases, attrs):
            if '__get__' in attrs:
                attrs['__get__'] = decorate_get(attrs['__get__'])
            return type.__new__(cls, name, bases, attrs)

Pentahedron

Sometimes you want the descriptor object? That violates the core expectation for descriptors: they're supposed to look like attributes. Why break that fundamental expectation? Why do this? Why create something so complex? – Borate

What I'm doing doesn't feel that complex to me, but I guess you could say I'm experimenting with the design. In my current, particular case, I have a descriptor that returns the strength of a weapon in a game. That value is a function of the descriptor's state (strength of the weapon) and the instance (health of the ship). There are different kinds of weapon; usually I just want the value result, but in a few cases, I need to know what kind of weapon it is -- the type of the descriptor. And what if a descriptor has methods not part of the descriptor protocol, and you want to call them? – Pentahedron

Answer 1

Most descriptors do their job when accessed as instance attribute only. So it's convenient to return itself when it's accessed for class:

class FixedValueProperty(object):
    def __init__(self, value):
        self.value = value
    def __get__(self, inst, cls):
        if inst is None:
            return self
        return self.value

This allows you to get descriptor itself:

>>> class C(object):
...     prop = FixedValueProperty('abc')
... 
>>> o = C()
>>> o.prop
'abc'
>>> C.prop
<__main__.FixedValueProperty object at 0xb7eb290c>
>>> C.prop.value
'abc'
>>> type(o).prop.value
'abc'

Note, that this works for (most?) built-in descriptors too:

>>> class C(object):
...     @property
...     def prop(self):
...         return 'abc'
... 
>>> C.prop
<property object at 0xb7eb0b6c>
>>> C.prop.fget
<function prop at 0xb7ea36f4>

Accessing descriptor could be useful when you need to extent it in subclass, but there is a better way to do this.

Merkley

Didn't realize that; good to know. Functions themselves would be an exception an exception to that pattern, but maybe the only one. Will have to poke at the built-in descriptors. – Pentahedron

Though it didn't answer my question as asked exactly, your answer did help me solve my underlying problem. I'll accept it. – Pentahedron

are functions an exception to this pattern (I assume you speak about methods)? No, c.method returns a bound method from a description, while C.method return an unbound method. It's the same pattern. – Nealon

Functions don't follow this pattern exactly, but very similar: function.__get__(None, cls) returns unbound method, not itself. – Merkley

staticmethod and classmethod also break this. – Jenson

Answer 2

The inspect library provides a function to retrieve an attribute without any descriptor magic: inspect.getattr_static.

Documentation: https://docs.python.org/3/library/inspect.html#fetching-attributes-statically

(This is an old question, but I keep coming across it when trying to remember how to do this, so I'm posting this answer so I can find it again!)

Argus

Answer 3

The above method

class FixedValueProperty(object):
    def __init__(self, value):
        self.value = value
    def __get__(self, inst, cls):
        if inst is None:
            return self
        return self.value

Is a great method whenever you control the code of the property, but there are some cases, such as when the property is part of a library controlled by someone else, where another approach is useful. This alternative approach can also be useful in other situations such as implementing object mapping, walking a name-space as described in the question, or other specialised libraries.

Consider a class with a simple property:

class ClassWithProp:

    @property
    def value(self):
        return 3
>>>test=ClassWithProp()
>>>test.value
3
>>>test.__class__.__dict__.['value']
<property object at 0x00000216A39D0778>

When accessed from the container objects class dict, the 'descriptor magic' is bypassed. Note also that if we assign the property to a new class variable, it behaves just like the original with 'descriptor magic', but if assigned to an instance variable, the property behaves as any normal object and also bypasses 'descriptor magic'.

>>> test.__class__.classvar =  test.__class__.__dict__['value']
>>> test.classvar
3
>>> test.instvar = test.__class__.__dict__['value']
>>> test.instvar
<property object at 0x00000216A39D0778>

Osman

Answer 4

Let's say we want to get the descriptor for obj.prop where type(obj) is C.

C.prop usually works because the descriptor usually returns itself when accessed via C (i.e., bound to C). But C.prop may trigger a descriptor in its metaclass. If prop were not present in obj, obj.prop would raise AttributeError while C.prop might not. So it's better to use inspect.getattr_static(obj, 'prop').

If you are not satisfied with that, here's a CPython-specific method (from _PyObject_GenericGetAttrWithDict in Objects/object.c):

import ctypes, _ctypes

_PyType_Lookup = ctypes.pythonapi._PyType_Lookup
_PyType_Lookup.argtypes = (ctypes.py_object, ctypes.py_object)
_PyType_Lookup.restype = ctypes.c_void_p

def type_lookup(ty, name):
    """look for a name through the MRO of a type."""
    if not isinstance(ty, type):
        raise TypeError('ty must be a type')

    result = _PyType_Lookup(ty, name)
    if result is None:
        raise AttributeError(name)

    return _ctypes.PyObj_FromPtr(result)

type_lookup(type(obj), 'prop') returns the descriptor in the same way when CPython uses it at obj.prop if obj is a usual object (not class, for example).

Ammoniacal