How to make len() work with different methods on different instances of a class, without modifying the class?

Question

Is there a way to make len() work with instance methods without modifying the class?

Example of my problem:

>>> class A(object):
...     pass
...
>>> a = A()
>>> a.__len__ = lambda: 2
>>> a.__len__()
2
>>> len(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: object of type 'A' has no len()

Note:

• different instances of A will have different __len__ methods attached
• I cannot change the class A

Guntar

Your problem is when you're defining __len__, try defining it as an actual method on the class instead of an attribute of an instance. – Dine

@PadraicCunningham See my modified question. This does not allow different __len__ methods for different instances of A. – Guntar

@ARF, I presume you cannot modify the class? – Pedigree

@PadraicCunningham You are right, I cannot modify the class. Based on the answers I am getting, I now see that I really should have mentioned this. – Guntar

@ARF: That sounds like you should either have your own wrapper type that contains an A object, or you should inherit from A. I recommend the wrapper; inheritance is deceptively tricky, especially when the superclass isn't designed for it. – Bolter

Can you shed some light on what kind of trick is used for ? – Fabrication

Why would you ever wan to do this? Is it for testing, or some other reason? Given you can't modify the class, why don't you use a wrapper type that modifies inst.__len__? – Rusk

Answer 1

It is actually possible without modifying the class based on this answer by Alex Martelli:

class A(object):
    def __init__(self, words):
        self.words = words

    def get_words(self):
        return self.words


a = A("I am a")
b = A("I am b")

def make_meth(inst, _cls, meth, lm):
    inst.__class__ = type(_cls.__name__, (_cls,), {meth: lm})

make_meth(a, A, "__len__", lambda self: 12)
make_meth(b, A, "__len__", lambda self: 44)

print(len(b))
print(len(a))
print(a.get_words())
print(b.get_words())

If we run the code:

In [15]: a = A("I am a")    
In [16]: b = A("I am b")    
In [17]: make_meth(a, A, "__len__", lambda self: 12)
In [18]: make_meth(b, A, "__len__", lambda self: 44) 
In [19]: print(len(b))
44    
In [20]: print(len(a))
12

In [21]: print(a.get_words())
I am a    
In [22]: print(b.get_words())
I an b

As per the last part of the last part of the linked answer, you can add any methods on a per instance basis using inst.specialmethod once you have used inst.__class__ = type(... :

In [34]: b.__class__.__str__ =  lambda self: "In b"

In [35]: print(str(a))
<__main__.A object at 0x7f37390ae128>

In [36]: print(str(b))
In b

Pedigree

Reassigning the object's class? I suppose it should work, as long as instances of A have a __dict__ (otherwise you get a TypeError). It has a few other implications likely to be confusing to anyone who has to deal with an object modified this way, but perhaps not substantially more confusing than changing the behavior of __len__ on someone else's class already is. – Bolter

@user2357112, yes , it works without changing the class which was a requirement of the OP's that they did not add to the question, I did not say it was the best solution but it shows that it is possible. – Pedigree

@ARF, it should be print(a.get_words()) with an s. I edited the answer and forgot to add the s – Pedigree

Am I understanding this right: this is basically a dynamic/implicit form of the wrapper-class solution @Bolter has suggested? (Only that the wrapper class is created after instantiation of the object rather than before.) – Guntar

It's not clear why you take _cls as a parameter here instead of using inst.__class__, and _cls.__class__.__name__ is the metaclass name instead of the class name (which is why ARF's error message was saying 'type' object instead of 'A' object). – Bolter

@ARF, no worries, it was my mistake. – Pedigree

@user2357112, because it was an oversight. I take _cls as the third argument is the class and it keeps the logic the same as Alex Martelli's example. Can you expand on perhaps not substantially more confusing than changing the behaviour of len on someone else's class already is? The question is about changing the behaviour or adding if it does not exist on a per instance basis which is what is happening, any other instances are not affected so not sure I understand what you are getting at unless I am missing something, nothing else should be affected? – Pedigree

@Guntar and Padraic: The "other implications" are a variety of subtle headaches that may or may not come up. For example, if A inherits object.__repr__, then objects modified this way will show up as <yourmodule.A object at ...> instead of <othermodule.A object at ...>. Anyone testing the type of these objects rather than isinstance (possibly for good reason, e.g. optimizations that shouldn't be applied to subclass instances) could be surprised, though probably not any more surprised than they would be by getting weird len results. – Bolter

@PadraicCunningham: Oh, are you asking about the "someone else's class" bit? I'm not saying that there are class-wide changes involved. It's just saying that if these objects are instances of a class you don't control (as they probably are if you can't change the __len__ implementation), then changing how they respond to len is probably going to cause a lot of surprises, more than the "other implications" of reassigning the class. – Bolter

@user2357112, yes, that was what I meant. I understand your points but I think this is a very particular use case, the len even if defined is going to be modified per instance so there is no real idea of a len method in the class, it may as well not be defined as whatever the class definition is will most likely not be match what will actually be happening. If you want a way to do it without changing the class definition then I don't imagine there are many other alternatives in the same style whether pretty or not. – Pedigree

@ARF, as far as anything going wrong I don't see much issues bar the slots issue already mentioned but if that were the case then a.anything = ... would have failed with an AttributeError. – Pedigree

There are a bunch more "subtle headaches" as yet unenumerated here. For example, type(a) != A, type(b) != A, type(a) != type(b). Also, all three classes have separate class dicts; assignments to A will be visible in a and b unless they occlude it (A.foo = 3; type(b).foo = 5; hasattr(b, "foo") == False). You could fix that by hand, type(a).__dict__ = A.__dict__ etc. – Baggywrinkle

make_meth: are you Walter White ? – Iva

Answer 2

No. Python always looks up special methods through the object's class. There are several good reasons for this, one being that repr(A) should use type(A).__repr__ instead of A.__repr__, which is intended to handle instances of A instead of the A class itself.

If you want different instances of A to compute their len differently, consider having __len__ delegate to another method:

class A(object):
    def __len__(self):
        return self._len()

a = A()
a._len = lambda: 2

Bolter

Good answer, takes into account OPs requirement of having different instances use a different method of determining len. Though why that is a requirement is beyond me... – Dine

Any method whose name begins and ends with __ would require this treatment. Because of the double underscores, these are sometimes called "dunder" methods. "Dirty deeds and their dunder chief!" – Christiansen

Answer 3

Special methods such as __len__ (double-underscore or "dunder" methods) must be defined on the class. They won't work if only defined on the instance.

It is possible to define non-dunder methods on an instance. However, you must convert your function to an instance method by adding a wrapper to it, which is how self gets passed in. (This would normally be done when accessing the method, as a method defined on the class is a descriptor that returns a wrapper.) This can be done as follows:

a.len = (lambda self: 2).__get__(a, type(a))

Combining these ideas, we can write a __len__() on the class that delegates to a len() that we can define on the instance:

class A(object):
     def __len__(self):
         return self.len()

a = A()
a.len = (lambda self: 2).__get__(a, type(a))

print(len(a))  # prints 2

You can actually simplify this in your case because you don't need self in order to return your constant 2. So you can just assign a.len = lambda: 2. However, if you need self, then you need to make the method wrapper.

Christiansen

Can you explain a bit more regarding the wrapper? @Bolter answer seems to give me access to self of the instance even without the wrapper. – Guntar

@ARF: No, I just left that out because your example lambda doesn't actually try to access self. If you do need self, you'd need to either get at it a different way (perhaps with a closure variable or functools.partial), or you'd need to do the __get__ thing. – Bolter

@Bolter Thanks, after some playing around with it, I now understand the wrapper issue. – Guntar

Answer 4

You have to define special methods at definition time of the class:

class A(object):
    def __len__(self):
        return self.get_len()

a = A()
a.get_len = lambda: 2
print len(a)

Systemic

It is false that "You have to define special methods at definition time of the class". class C: pass followed by c=C() C.__len__ = lambda self: 2 print(len(c)) works – Upheaval

@TerryJanReedy: The OP uses new-style classes. You use old-style classes. So what's the point? – Systemic

@Systemic Old style classes do not even exist in 3.x, which is what I used. – Upheaval

Answer 5

You did not specify whether you're using Python 2.x or 3.x, and in this case it depends! In 3.x, your question has already been answered by other people. In 2.x, your question actually has an easy answer: stop deriving from object! (Of course, if you follow this approach, you can't upgrade to 3.x until you find another way to do this, so don't do what I'm suggesting unless you have no plans to upgrade any time soon.)

Type "copyright", "credits" or "license()" for more information.
>>> class A:pass

>>> a=A()
>>> a.__len__ = lambda: 2
>>> len(a)
2
>>> 

Enjoy :)

Chrysanthemum