id()s of bound and unbound method objects --- sometimes the same for different objects, sometimes different for the same object [duplicate]

Question

I've tried some code about bound and unbound methods. When we call them, I think both of them would return objects. But when I use id() for getting some information, it returns something I don't understand.

IDE: Eclipse

Plugin: pydev

Class C(object):
    def foo(self):
        pass

cobj = C()

print id(C.foo)    #1
print id(cobj.foo) #2

a = C.foo
b = cobj.foo

print id(a)        #3
print id(b)        #4

And the output is...

5671672

5671672

5671672

5669368

Why do #1 and #2 return the same id? Aren't they different objects? And if we assign C.foo and conj.foo to two variables, #3 and #4 return the different id.

I think #3 and #4 show that they are not the same object, but #1 and #2...

What is the difference between the id of bound method, and an unbound method?

Slabber

Answer 1

Whenever you look up a method via instance.name (and in Python 2, class.name), the method object is created a-new. Python uses the descriptor protocol to wrap the function in a method object each time.

So, when you look up id(C.foo), a new method object is created, you retrieve its id (a memory address), then discard the method object again. Then you look up id(cobj.foo), a new method object created that re-uses the now freed memory address and you see the same value. The method is then, again, discarded (garbage collected as the reference count drops to 0).

Next, you stored a reference to the C.foo unbound method in a variable. Now the memory address is not freed (the reference count is 1, instead of 0), and you create a second method instance by looking up cobj.foo which has to use a new memory location. Thus you get two different values.

See the documentation for id():

Return the “identity” of an object. This is an integer (or long integer) which is guaranteed to be unique and constant for this object during its lifetime. Two objects with non-overlapping lifetimes may have the same id() value.

CPython implementation detail: This is the address of the object in memory.

Emphasis mine.

You can re-create a method using a direct reference to the function via the __dict__ attribute of the class, then calling the __get__ descriptor method:

>>> class C(object):
...     def foo(self):
...         pass
... 
>>> C.foo
<unbound method C.foo>
>>> C.__dict__['foo']
<function foo at 0x1088cc488>
>>> C.__dict__['foo'].__get__(None, C)
<unbound method C.foo>
>>> C.__dict__['foo'].__get__(C(), C)
<bound method C.foo of <__main__.C object at 0x1088d6f90>>

Note that in Python 3, the whole unbound / bound method distinction has been dropped; you get a function where before you'd get an unbound method, and a method otherwise, where a method is always bound:

>>> C.foo
<function C.foo at 0x10bc48dd0>
>>> C.foo.__get__(None, C)
<function C.foo at 0x10bc48dd0>
>>> C.foo.__get__(C(), C)
<bound method C.foo of <__main__.C object at 0x10bc65150>>

Furthermore, Python 3.7 adds a new LOAD_METHOD - CALL_METHOD opcode pair that replaces the current LOAD_ATTRIBUTE - CALL_FUNCTION opcode pair precisely to avoid creating a new method object each time. This optimisation transforms the executon path for instance.foo() from type(instance).__dict__['foo'].__get__(instance, type(instance))() with type(instance).__dict__['foo'](instance), so 'manually' passing in the instance directly to the function object.

Mistrust

@AshwiniChaudhary: It's an implementation detail of CPython; other python implementations probably use a different value for id(). – Mistrust

Possibly nicer on the eyes: MethodType(vars(C)['foo'], C(), C). But it's not as if code like this should be commonplace. – Robenarobenia

@eryksun: My point was to illustrate how python creates the method in the first place; to show that it is not created once for a class then retrieved each time. Using types.MethodType() would defeat that purpose. – Mistrust

I appreciate that and upvoted your answer. I was just offering up an alternative to directly using the special attributes. – Robenarobenia

@eryksun: Sure; still, I'd probably still use __get__ myself to bind a function to an instance (e.g. func.__get__(instance, type(instance))); to me personally that documents the purpose quite well. – Mistrust

I never knew method 'objects' are created every time they are called. Is there any advantage to this technique? Why not keeping them in the object's dict? – Redolent

@Vinny: because Python is highly dynamic. instance.method() can result to a completely different function object from one call to the next. – Mistrust

@Vinny: Python 3.7 does now avoid actually creating the method object in the most common cases thanks to two new bytecodes; this applies when there is an attribute lookup with a call directly following in the same expression. I already cover that in my answer here. – Mistrust

@TomKarzes yes, which is why I stated that in my answer. – Mistrust

@MartijnPieters Ah ok, I hadn't read far enough. I had encountered this in Python 3, when I saw that id(cobj.foo) == id(cobj.foo). But this was because the object created by cobj.foo was being freed after being passed to the first id call, then the same id was being reused by the second cobj.foo before it was passed to the second id call. If the bound methods are both live, this doesn't happen, e.g. m1 = cobj.foo; m2 = cobj.foo; id(m1) == id(m2) is guaranteed to be False. – Frap

@TomKarzes yes, I cover those aspects in my answers on string interning, and how unique the id() value is. – Mistrust

Answer 2

Adding to @Martijn Pieters's very good answer:

In [1]: class C(object):
   ...:     def foo(self):
   ...:         pass
   ...:

In [2]: c = C()

In [3]: id(c.foo), id(C.foo)
Out[3]: (149751844, 149751844)  # so 149751844 is current free memory address

In [4]: a = c.foo  # now 149751844 is assigned to a

In [5]: id(a)              
Out[5]: 149751844

# now python will allocate some different address to c.foo and C.foo     

In [6]: id(c.foo), id(C.foo)    # different address used this time, and
Out[6]: (149752284, 149752284)  # that address is freed after this step

# now 149752284 is again free, as it was not allocated to any variable

In [7]: b = C.foo  # now 149752284 is allocated to b    

In [8]: id(b)
Out[8]: 149752284                

In [9]: c.foo is C.foo  # better use `is` to compare objects, rather than id()
Out[9]: False

Toul