What's __new__ by default in Python 3?

Question

I believe I have some sort of understanding of what __new__ is supposed to do (create an instance, of a class, but not initialize it, that is the job of __init__). I'd like to understand, however, what Python 3 implements as a __new__ method by default.

I also find it somewhat confusing that cls is an argument for __new__, but __new__ is a staticmethod and not a classmethod (I got this from the documentation). How does it know that it is being passed a type as its first argument?

Sinclair

1. object.__new__ is the default, what is to understand about it? 2. By checking isinstance(cls, type) or the equivalent in the language of the implementation. – Ochlocracy

Here it is: github.com/python/cpython/blob/… – Soren

@Soren I never went into the CPython implementation, can you explain what is going on in there? it looks like something analogous to self.__init__() is returned? – Sinclair

@Goyo What do you mean? If object.__new__ is the default, what does it do? Where can I find this info? Also, what about cls as an argument? Is __new__ a classmethod? – Sinclair

@Soren can this be recreated in pure python or is this not possible? – Sinclair

@Sinclair possibly using ctypes, but why would you want to? – Soren

@Soren I'm not interested in doing this, I just want to understand what is happening under the hood when I don't override __new__ (preferably without going into CPython). – Sinclair

The actual object is created, i.e. some memory is allocated on the private heap. A reference to that memory is returned – Soren

It creates a new instance and returns a reference to it, but you already know that. The details are implementation-dependant but you are not interested in implementation details. So what is what you want to know? – Ochlocracy

__new__ is a static method, that is already in your question too. – Ochlocracy

Answer 1

First part: what does __new__ do by default? Because the creation of an object from scratch is a fundamental, implementation-specific operation, the definition of object.__new__ is necessarily (like the rest of the definition of object) part of the implementation itself. That means you need to look in the source code of CPython, PyPy, Cython, etc. to see exactly how object creation is managed in any particular implementation of Python. Typically, it's all low-level bookkeeping that can't be accessed directly from Python itself.

Second part: how does __new__ know that it gets a class argument? Because it assumes its first argument is a class, and the caller had better provide a class if they expect __new__ to work correctly! That said, nobody really ever calls __new__ expclitly, except via super in an override of __new__, and then, you have to make sure to pass cls explicitly yourself:

def __new__(cls, *args, **kwargs):
    rv = super().__new__(cls, *args, **kwargs)  # Not super().__new__(*args, **kwargs)

The rest of the time, you create an instance of a class Foo not by calling Foo.__new__(Foo, ...) directly, but just by calling the type itself: Foo(...). This is managed because the __call__ method of Foo's metaclass takes care of calling __new__ for your. For example, you can imagine that type.__call__ is defined roughly as

# A regular instance method of type; we use cls instead of self to
# emphasize that an instance of a metaclass is a class
def __call__(cls, *args, **kwargs):
    rv = cls.__new__(cls, *args, **kwargs)  # Because __new__ is static!
    if isinstance(rv, cls):
        rv.__init__(*args, **kwargs)
    return rv

Note that __init__ is only invoked if __new__ actually returns an instance of the class calling __new__ in the first place.

Cupboard

Ahhhh, I see now that this is what I was looking for. Your rough idea of how __call__ works made things click for me. I think in this context I can understand much better what __new__ does. – Sinclair

Answer 2

Based on my understanding the default implementation of __new__() is something like this

class Default(object):
def __new__(cls, *args, **kwargs):
    print("In new")
    return super().__new__(cls,*args, **kwargs)

def __init__(self):
    print("In init default")

default = Default()
print(type(default))

Output

In new
In init default
<class '__main__.Default'>

Based on the documentation https://docs.python.org/3/reference/datamodel.html#object.new

Typical implementations create a new instance of the class by invoking the superclass’s __new__() method using super().new(cls[, ...]) with appropriate arguments and then modifying the newly-created instance as necessary before returning it.

The super().__new__() call will create the instance and __init__() will initialize.

Below code shows an incorrect overriding of __new__()

class InccorectOverride(object):

def __new__(cls, *args, **kwargs):
    pass

def __init__(self):
    print("In init InccorectOverride")

a = InccorectOverride()
print(type(a))

Output

<class 'NoneType'>

Since __new__() does not return the instance, the output has a value of NoneType

Hope this answers your question

Nondescript

Answer 3

Your question was

If we do not override it, then what does __new__ do in Python?

---

Suppose that we have a very simple class named Point. We instantiate the Point class as follows:

# construct and initialize paula the point
paula = Point(10.2, 7.4)

In order to help explain what __new__ does, I will write a classmethod named make_point which has almost the same behavior as calls to the class constructor, such as paula = = Point(10.2, 7.4).

class Point():
    def __new__(cls, *args, **kwargs):
        print("__new__(" + ", ".join(str(x) for x in [cls, *args]) + ")")
        obj = super().__new__(cls)
        return obj

    def __init__(self, y:float, y:float):
        args = (y, y)
        print("__init__(" + ", ".join(str(x) for x in [self, *args]) + ")")
        self._y = y
        self._y = y

    @classmethod
    def make_point(cls, *args):
        new_instance = cls.__new__(cls, *args)
        if isinstance(new_instance, cls):
            cls.__init__(new_instance, *args)
        return new_instance

Now we can instantiate the Point class as follows:

peter = Point.make_point(59.87, 5.91)

Neuroma