Why magic methods are not intercepted by __getattr__ implementation in python?

Question

Subj.

Example:

MyList(object):

    def __init__(self, list_to_be_wrapped):
          self._items = list_to_be_wrapped

    def __getattr__(self, item):
          return getattr(self._items, item)

Then

MyList([1, 2, 3])[-1]

will raise: TypeError: ‘MyList’ object does not support indexing

while:

MyList([1, 2, 3]).pop()

will work perfectly (pop will be intercepted by the __getattr__ and redirected to _items)

It seems for me that interception of magic methods by __getattr__ would be very helpful considering the case of "implementing own container types via composition not inheritance". Why is such "interception" not supported in python?

Kirmess

getitem is used under the hood when calling indexes on objects. def __getitem__(self, y): """ x.__getitem__(y) <==> x[y] """ – Kirmess

I think the OP's question is why magic methods aren't redirected to __getattr__. In other words, why is obj[x] not equivalent to obj.__getattr__("__getitem__")(x). To the OP: If that were the case, it would imply that __getattr__ would have to be sent to itself ;) – Horn

@CeasarBautista Indeed it does, see my answer below. – Horn

Answer 1

For the full answer, you need to look at the documentation for special method lookup, specifically:

In addition to bypassing any instance attributes in the interest of correctness, implicit special method lookup generally also bypasses the __getattribute__() method even of the object’s metaclass.

Which is justified by:

Bypassing the __getattribute__() machinery in this fashion provides significant scope for speed optimisations within the interpreter, at the cost of some flexibility in the handling of special methods (the special method must be set on the class object itself in order to be consistently invoked by the interpreter).

Note that this only applies for new-style classes (classes which extend object), but you really should always use new-style classes for any new code.

The simplest solution is that you need to manually proxy all of the special methods you wish to redirect to the proxy'd class.

Horn

It's interesting to note that this only applies to new-style classes. If you don't subclass from object, OP's code actually works. – Mossgrown

@aruisdante, Thank you for advice to "manually proxy all..." ... Currently I am looking how to "automate" thise proxying... Something like the following: setattr(self, '__getitem__', lambda *args: getattr(self._items, '__getitem__')(*args)) inside __init__ method does not work... Is there any way to achieve what i want? What did I wrong in my solution? – Kirmess

@Kirmess here is a really good recipe for general object proxying in Python, as well as a good SO QA on it: #9943036. The recipe shows you about the limit to how much you can automate the proxying facilities. – Horn