How does python close files that have been gc'ed?

Question

I had always assumed that a file would leak if it was opened without being closed, but I just verified that if I enter the following lines of code, the file will close:

>>> f = open('somefile.txt')
>>> del f

Just out of sheer curiosity, how does this work? I notice that file doesn't include a __del__ method.

Search

Answer 1

In CPython, at least, files are closed when the file object is deallocated. See the file_dealloc function in Objects/fileobject.c in the CPython source. Dealloc methods are sort-of like __del__ for C types, except without some of the problems inherent to __del__.

Whitehead

To clarify, del is called during garbage collection, and in the C implementation of Python for file objects that occurs the moment when there are no more references to the file object. – Mooned

Answer 2

Hence the with statement.

For Python 2.5, use

from __future__ import with_statement

(For Python 2.6 or 3.x, do nothing)

with open( "someFile", "rU" ) as aFile:
    # process the file
    pass
# At this point, the file was closed by the with statement.
# Bonus, it's also out of scope of the with statement,
# and eligible for GC.

Stumpage

That's what I assumed as well. But on OS X in Python 2.5.1, the lines of code that I posted causes the Python interpreter to release the file (verified in the Activity Monitor). – Search

Python is supposed to close the file when it's collected. I've been looking through for where this happens in fileobject.c, but it's not there. It's probably somewhere in the gc mechanism, which is where I'm looking next. I like this question. – Rufous

Looks like I missed it in fileobject.c (see Gallagher). I really wish I had a better understanding of the CPython internals. – Rufous

It's not in the gc module. The C implementation of Python uses reference counting, and the code Devin is looking for is in object.h in Py_DECREF, Py_XDECREF, and Py_DecRef. The "del f" releases the last reference, which triggers the _Py_Dealloc call. – Mooned

Answer 3

Python uses reference counting and deterministic destruction in addition to garbage collection. When there is no more references to an object, the object is released immediately. Releasing a file closes it.

This is different than e.g. Java where there is only nondeterministic garbage collection. This means you connot know when the object is released, so you will have to close the file manually.

Note that reference counting is not perfect. You can have objects with circular references, which is not reachable from the progam. Thats why Python has garbage collection in addition to reference counting.

Codeine

Answer 4

Best guess is that because the file type is a built-in type, the interpreter itself handles closing the file on garbage collection.

Alternatively, you are only checking after the python interpreter has exited, and all "leaked" file handles are closed anyways.

Bridgeman

My understanding of "primitive type" (received from Java) doesn't leave file as a primitive, since there are no primitives in Python. – Rufous

I think HUAGHAGUAH meant to say "built-in type." :) – Search