Python del statement

Question

Calling del on a variable in Python. Does this free the allocated memory immediately or still waiting for garbage collector to collect? Like in java, explicitly calling del has no effect on when the memory will be freed.

Ramsgate

The point of garbage collection is that you don't have to worry about when the memory is freed. So why are you worrying about it? – Dallman

I'm processing large volume of traffic and observed memory leak issues. I'm almost certain the issue is not in the python script, but just trying to make sure. – Ramsgate

Answer 1

The del statement doesn't reclaim memory. It removes a reference, which decrements the reference count on the value. If the count is zero, the memory can be reclaimed. CPython will reclaim the memory immediately, there's no need to wait for the garbage collector to run.

In fact, the garbage collector is only needed for reclaiming cyclic structures.

As Waleed Khan says in his comment, Python memory management just works, you don't have to worry about it.

Dispend

I don't believe CPython ever frees the memory back to the OS, even after it's been reclaimed. – Auriga

So when you say "If the count is zero, the memory can be reclaimed.", do you mean that it still needs to wait for garbage collector to be "actually" reclaimed or? – Ramsgate

I'm not sure how else to say it: "CPython will reclaim the memory immediately, there's no need to wait for the garbage collector to run." – Dispend

That comment is in contradiction with a lot of what I've seen around SO today, but reading a large database table in to memory as a copy and then del reference causes the memory to be freed, so garbage collection is getting called, and (though there are no time guarantees on this) the interpreter is giving that memory back to the system. If I instead read my table and don't assign it to a variable, then it is an island, and I have to call gc.collect() explicitly to find it and free. – Roncesvalles

@pvlkmrv That doesn't sound right: If you don't assign the table to a variable, then the reference count will be zero, and it will be reclaimed. Islands aren't a problem, it's circular references that require a gc pass to reclaim. – Dispend

Empirical observation shows that creating a large list in-line and passing it as an indexing object to a numpy array causes that list to hang around in memory even after I del the reference to the returned sub-array and Python decides in its own time (usually a second or so) to give that memory back to the system. Passing a numpy array as the indexing object, I observe the memory of that object is freed after the sub-array is returned. – Roncesvalles

Likewise however I read a table (in a terminal, mind you), if I don't assign it to a variable, it does take memory and stick there even though I have no reference to it. – Roncesvalles

If you do the same thing twice: A) assigning the result to a variable, and B) not assigning it to a variable, there is no way that case B will take longer to reclaim the memory than case A will. I'm not sure what you are observing, but the delay is not because the value isn't assigned to a variable. – Dispend

The bottom line is that, del variable seems to cascade in to a garbage collection of that object rather immediately if the refcount goes to zero, and in cases where the interpreter has malloced large sections of memory with mmap rather than on the heap with brk/sbrk it seems to return that memory to the system fairly immediately too. – Roncesvalles

del x doesn't do anything different for garbage collection than x = None does. It just removes a reference from x's value. – Dispend

del x would decrease the reference count by 1; x= None would decrease the reference count to zero. Is this no true? – Peasant

Answer 2

"Deletion of a name removes the binding of that name from the local or global namespace". No more, no less. It does nothing to the object the name pointed to, except decrementing its refcount, and if refcount is not zero, the object will not be collected even when GC runs.

Absher

Answer 3

Also, the del statement seems to be a little bit faster than assigning None (similar to Java's style assigning null to a variable to free its memory ...).

To compare:

import time, math

def measure_del():
        start = time.time()
        for i in range(0,int(math.pow(10,8))):
                    a = "123"
                    del a # <--- !!!
        end = time.time()
        print(end-start)

def measure_none():
        start = time.time()
        for i in range(0,int(math.pow(10,8))):
                    a = "123"
                    a = None # <--- !!!
        end = time.time()
        print(end-start)

results in (running in idle3.4):

>>> measure_del()
3.9930295944213867
>>> measure_del()
3.7402305603027344
>>> measure_del()
3.8423104286193848
>>> measure_del()
3.753770351409912
>>> measure_del()
3.7772741317749023
>>> measure_del()
3.815058946609497

>>> measure_none()
4.052351236343384
>>> measure_none()
4.130320072174072
>>> measure_none()
4.082390069961548
>>> measure_none()
4.100180625915527
>>> measure_none()
4.071730375289917
>>> measure_none()
4.136169672012329

Ence

Why are programmers so fascinated with the speed of things that make no difference to the overall speed of the program? :) – Dispend

Well, it's something like a pack of paper. E.g. you can buy 500 sheets with a grammage of 80g/m² or 90g/m². If you write on one single sheet, you'll hardly notice a difference. But if you compare both packages, you'll see a clear difference in height, weight and price. Writing good code, it is certainly not a bad idea to balance the readability of the code with the performance on machine level. – Ence

In some cases you want that speed, because you need to free some underlying C++ memory before running out of buffer for example. The exact case that made me read that question. – Oddball

Answer 4

Regarding delete: Sometimes you have to work on large datasets where you have to compute memory-intensive operations and store a large amount of data into a variable in a recursive manner. To save RAM, when you finish your entire operation, you should delete the variable if you are no more using it outside the recursive loop. You can use the command

del varname followed by Python’s garbage collector gc.collect()

Regarding speed: Speed is the most important in applications such as financial applications with a regulatory requirement. You have to make sure that the speed of operation is completed within the expected timeframe.

Hendry