The proper way of completely overriding attribute access in Python?

Question

This naive class attempts to mimic the attribute access of basic python objects. dict and cls explicitly stores the attributes and the class. The effect is that accessing .x of an instance will return dict[x], or if that fails, cls.x. Just like normal objects.

class Instance(object):
    __slots__ = ["dict", "cls"]
    def __getattribute__(self, key):
        try:
            return self.dict[key]
        except KeyError:
            return getattr(self.cls, key)
    def __setattr__(self, key, value):
        if key == "__class__":
            self.cls = value
        else:
            self.dict[key] = value

But it's nowhere near as simple as that. One obvious issue is the complete disregard for descriptors. Just imagine that cls has properties. Doing Instance.some_property = 10 should access the property as defined in cls, but will instead happily set some_property as an attribute in dict.

Then there is the issue of binding methods of cls to instances of Instance, and possibly more that I don't even know.

There seem to be a lot of details to get the above class to function as close to python objects as possible, and the docs for descriptors I've read so far hasn't made it clear how to get, simply put, everything right.

What I am asking for is a reference for implementing a complete replacement for python's attribute access. That is, the above class, but correct.

Detestation

You have obviously figured out, that the "dot access" of a property of an instance is not as easy as it looks like. The documentation of how it works is the source code. You just ask for a reimplementation. If you want to change something, please ask a more specific question. – Homogeneous

@Homogeneous You can't necessarily use the same method CPython uses, since it depends on C-level access to Python objects. Maybe a look at the source of the PyPy Python implementation would be helpful, since they already implement true Python attribute access in (R)Python. – Decorator

What on Earth are you hoping to accomplish with this? – Rosmunda

@Achim: I don't want to change anything beyond redirecting the attribute access to the specified dict and cls. Hence why the question asks for how to do it the standard way. @agf: The PyPy suggestion might be a good place to start, thanks. Though I am still looking for an answer to this question. @Karl Knechtel: The reason to ask something on SO is generally because I assume someone else will be able to answer with less effort than it would take me to research it. – Detestation

Answer 1

Well, I needed this answer so I had to do the research. The below code covers the following:

• data-descriptors are given precedence both when setting and getting attributes.
• non-data descriptors are properly called in __getattribute__

There may be typos in the code below as I had to translate it from an internal project. And I am not sure if it is 100% like python objects, so if anyone could spot errors that would be great.

_sentinel = object()

def find_classattr(cls, key):
  for base in cls.__mro__: # Using __mro__ for speed.
    try: return base.__dict__[key]
    except KeyError: pass
  return _sentinel

class Instance(object):
  __slots__ = ["dict", "cls"]
  def __init__(self, d, cls):
    object.__setattr__(self, "dict", d)
    object.__setattr__(self, "cls", cls)
  def __getattribute__(self, key):
    d = object.__getattribute__(self, "dict")
    cls = object.__getattribute__(self, "cls")
    if key == "__class__":
      return cls
    # Data descriptors in the class, defined by presence of '__set__',
    # overrides any other kind of attribute access.
    cls_attr = find_classattr(cls, key)
    if hasattr(cls_attr, '__set__'):
      return cls_attr.__get__(self, cls)
    # Next in order of precedence are instance attributes.
    try:
      return d[key]
    except KeyError:
      # Finally class attributes, that may or may not be non-data descriptors.
      if hasattr(cls_attr, "__get__"):
        return cls_attr.__get__(self, cls)
      if cls_attr is not _sentinel:
        return cls_attr

    raise AttributeError("'{}' object has no attribute '{}'".format(
      getattr(cls, '__name__', "?"), key))
  def __setattr__(self, key, value):
    d = object.__getattribute__(self, "dict")
    cls = object.__getattribute__(self, "cls")
    if key == "__class__":
      object.__setattr__(self, "cls", value)
      return

    # Again, data descriptors override instance attributes.
    cls_attr = find_classattr(cls, key)
    if hasattr(cls_attr, '__set__'):
      cls_attr.__set__(self, value)
    else:
      d[key] = value

Funny thing is I realized I had written exactly the same stuff before a couple of years ago, but the descriptor protocol is so arcane I had forgotten it since.

EDIT: Fixed bug where using getattr to find an attribute on the class would call it's descriptors on the class level (i.e. without the instance). Replaced it with a method that looks directly in the __dict__ of the bases.

Detestation

What if you want to get the fake instance dict from outside the class? You don't have an instance of cls here so what if methods you call on the class require setup done in its __new__ or __init__? – Decorator

It's no problem to call for example cls.__init__ at the end of Instance.__init__, neither should anyone (needing something this advanced) have trouble figuring out how to add access to the internal dict. Personally I use a factory that first creates an Instance and then calls __init__ with the supplied class. – Detestation