RAII in Python: What's the point of __del__?

Question

At first glance, it seems like Python's __del__ special method offers much the same advantages a destructor has in C++. But according to the Python documentation (https://docs.python.org/3.4/reference/datamodel.html), there is no guarantee that your object's __del__ method ever gets called at all!

It is not guaranteed that __del__() methods are called for objects that still exist when the interpreter exits.

So in other words, the method is useless! Isn't it? A hook function that may or may not get called really doesn't do much good, so __del__ offers nothing with regard to RAII. If I have some essential cleanup, I don't need it to run some of the time, oh, when ever the GC feels like it really, I need it to run reliably, deterministically and 100% of the time.

I know that Python provides context managers, which are far more useful for that task, but why was __del__ kept around at all? What's the point?

Twopenny

I don't really get what exactly you are asking about. Is your question "Is __del__ essentially useless?"? (I believe it is your question) In this case I believe you'd clarify the question by removing all references to C++ or RAII, which are completely unrelated. If your question is "How to implement RAII in Python? May I use __del__ for this?" then you probably have to change a lot of stuff in your question. – Burdelle

@Burdelle The crux of the matter is, what exactly is the point of ´__del__´? Can someone name a realistic use case for the method? – Twopenny

Answer 1

__del__ is a finalizer. It is not a destructor. Finalizers and destructors are entirely different animals.

Destructors are called reliably, and only exist in languages with deterministic memory management (such as C++). Python's context managers (the with statement) can achieve similar effects in certain circumstances. These are reliable because the lifespan of an object is precisely fixed; in C++, objects die when they are explicitly deleted or when some scope is exited (or when a smart pointer deletes them in response to its own destruction). And that's when destructors run.

Finalizers are not called reliably. The only valid use of a finalizer is as an emergency safety net (NB: this article is written from a .NET perspective, but the concepts translate reasonably well). For instance, the file objects returned by open() automatically close themselves when finalized. But you're still supposed to close them yourself (e.g. using the with statement). This is because the objects are destroyed dynamically by the garbage collector, which may or may not run right away, and with generational garbage collection, it may or may not collect some objects in any given pass. Since nobody knows what kinds of optimizations we might invent in the future, it's safest to assume that you just can't know when the garbage collector will get around to collecting your objects. That means you cannot rely on finalizers.

In the specific case of CPython, you get slightly stronger guarantees, thanks to the use of reference counting (which is far simpler and more predictable than garbage collection). If you can ensure that you never create a reference cycle involving a given object, that object's finalizer will be called at a predictable point (when the last reference dies). This is only true of CPython, the reference implementation, and not of PyPy, IronPython, Jython, or any other implementations.

Keikokeil

One of the quotes on the page you linked to is > A correctly-written program cannot assume that finalizers will ever run. This just reinforces my belief that they're essentially useless. Even your remark that they can be used as an emergency safety net to try and clean up stuff that the programmer forgot doesn't really change that because again, it may or may not happen. At any rate, I think this is a horrible idea. It just creates a false sense of security, creates confusion and encourages sloppy resource management (hey, I don't have to clean up, Python might just do it for me). – Twopenny

I happen to agree with you. That's why I never use finalizers. OTOH, the finalizers attached to weak references can occasionally be useful, if for example you need to maintain a collection of live objects but don't want to keep them alive. See weakref for more information. – Keikokeil

PyPy does guarantee your del will be called however. In Cpython if it's a part of cycle, it won't be called AT ALL. – Ranchero

@fijal: This is no longer true as of 3.4+. See PEP 442. Furthermore, PyPy does guarantee a call, but not at a predictable time, so it is insufficient for most purposes. – Keikokeil

Yes, I know about that PEP. For most purposes, del is indeed a bad idea, don't use it. CPython does not guarantee to have just one call either (in case of resurrection, del will be called repeatedly). I find "called once or more, likely straight after, but not completely in case of cycle" to be not a sufficient guarantee either :-) – Ranchero

@Ranchero From python3.4+ __del__ methods are guaranteed to be called only once, even if they are resurrected (read the PEP) – Burdelle

Answer 2

Because __del__ does get called. It's just that it's unclear when it will, because in CPython if you have circular references, the refcount mechanism can't take care of the object reclamation (and thus its finalization via __del__) and must delegate it to the garbage collector.

The garbage collector then has a problem: he cannot know in which order to break the circular references, because this may trigger additional problems (e.g. frees the memory that is going to be needed in the finalization of another object that is part of the collected loop, triggering a segfault).

The point you stress is because the interpreter may exit for reasons that prevents it to perform the cleanup (e.g. it segfaults, or some C module impolitely calls exit() ).

There's PEP 442 for safe object finalization that has been finalized in 3.4. I suggest you take a look at it.

https://www.python.org/dev/peps/pep-0442/

Secco

Which again makes it clear how useless __del__ is. For instance, if I where to use objects of a mutex guard class, I couldn't do the unlocking in the guard class's __del__ method, because it's not enough to know that __del__ will get called when ever the GC feels like doing its job ... I need the mutex released right when the guard object "goes out of scope" (I know that's stretching it, given Python's scoping rules). IMO __del__ gives programmers the false impression that RAII in Python is doable via destructors when it really isn't. You have to use context managers for that. – Twopenny

It's not useless. You are trying to code C++ in python. The point is that __del__ is not a destructor. It's a mechanism that is in place and may be useful in very limited cases, but you can't rely on it, akin to the java finalizer. See this thread for a similar question #158674 – Secco