Why isn't __new__ in Python new-style classes a class method?

Question

The Changelog for Python 2.2 (where new-style classes were introduced) says the following about the __new__ function:

__new__ is a static method, not a class method. I initially thought it would have to be a class method, and that's why I added the classmethod primitive. Unfortunately, with class methods, upcalls don't work right in this case, so I had to make it a static method with an explicit class as its first argument.

However, I cannot think of why class methods wouldn't work for this purpose, and it would certainly look better. Why didn't __new__ end up as a class method in the end? What does Guido refer to when he says that "upcalls don't work right in this case"?

Demagogy

Answer 1

__new__ being static method allows a use-case when you create an instance of a subclass in it:

return super(<currentclass>, cls).__new__(subcls, *args, **kwargs)

If new is a class method then the above is written as:

return super(<currentclass>, cls).new(*args, **kwargs)

and there is no place to put subcls.

I don't really see when that would be a proper use of __new__, though. Maybe I'm not seeing it, but that just seems to me to be a completely pathological use of it (and it should be said, that if you still really want it, then you could access it with object.__new__.__func__). At the very least, I find it very hard to imagine that it would have been the reason for Guido to change __new__ from being a class method to a static method.

A more common case would be to call parent __new__ without using super(). You need a place to pass cls explicitly in this case:

class Base(object):
    @classmethod
    def new(cls):
        print("Base.new(%r)" % (cls,))
        return cls()

class UseSuper(Base):
    @classmethod
    def new(cls):
        print("UseSuper.new(%r)" % (cls,))
        return super(UseSuper, cls).new() # passes cls as the first arg

class NoSuper(Base):
    @classmethod
    def new(cls):
        print("NoSuper.new(%r)" % (cls,))
        return Base.new()  # passes Base as the first arg

class UseFunc(Base):
    @classmethod
    def new(cls):
        print("UseFunc.new(%r)" % (cls,))
        return Base.new.im_func(cls)  # or `.__func__(cls)`. # passes cls as the first arg

print(UseSuper.new())
print('-'*60)
print(NoSuper.new())
print('-'*60)
print(UseFunc.new())

Sedlik

@Duncan: I've intentionally used new rather than __new__. If it is unclear, leave a comment I'll try to elaborate. – Sedlik

I think if you had used a name that was a bit more distinct than just dropping the underscores it would have been clearer, but I understand now thanks. – Metasomatism

When would this actually be a problem, though? If you want an instance of subcls, why wouldn't you call subcls.new()? Doing it the way you describe only has the effect of not running the proper __new__ functions on subcls. – Demagogy

@Dolda2000: the point is that it creates a subclass instance using code from a parent __new__(). And __new__ being a @classmethod would complicate that. I can't think of a real use case from the top of my head when you need it i.e., you've called Base() but got Derived(). If you exclude subclasses of immutable types then overriding __new__ is an esoteric language feature (like metaclasses) that might be useful in equally esoteric cases. – Sedlik

I don't really see when that would be a proper use of __new__, though. Maybe I'm not seeing it, but that just seems to me to be a completely pathological use of it (and it should be said, that if you still really want it, then you could access it with object.__new__.__func__). At the very least, I find it very hard to imagine that it would have been the reason for Guido to change __new__ from being a class method to a static method. – Demagogy

@Dolda2000: A more common case would be to call parent __new__ without using super(). You need a place to pass cls explicitly in this case. – Sedlik

Your last comment seems to me to be the very reason why BDFL decided to make __new__ a static method (not a class method). – Rascon

That's a good point. I can see how that could be the actual reason, yes. – Demagogy

@Sedlik I miss read a previous comment and responded. I’ve deleted the comment. – Hegelianism

@jfs: I can not understand the example you gave: can __new__() be rewritten without preceding and trailing double underscores? .. and why do that at first place? (you said you could elaborate, please do) Thanks – Caveat

@Searcherer: if you need to pass a subclass to the super method, then no (without hacks). – Sedlik

@jfs: .. I can't relate. In your suggested example, you define new(), not __new__(), is this acceptable? Isn't Python expecting __new__(), not new()? – Caveat

@Searcherer: no. new() and __new__ are completely different methods. "new" here is an ordinary class method (it could have been any name e.g., "new_method_to_illustrate_some_point_in_the_answer" <-- method name). __new__ is magic (static) method -- it must be called __new__. – Sedlik

@jfs: Thank you. – Caveat

"A more common case would be to call parent new without using super()" Why is this a "more common case"? This seems simply incorrect. You have no way of knowing that "Base" is your parent class. You must call super. – Mankind

@NeilG: "more common" is relative to the "use-case when you create an instance of a subclass in it." Note: the question is about the decision in Python 2.2 The answer uses Python 2. You can use super with static methods there but it may be surprising: #26788714 – Sedlik

@Sedlik I don't see why he couldn't have made it a class method with the additional class parameter to indicate which class should be constructed. This would have solved all of the problems, I think. – Mankind