`del` on a package has some kind of memory

Question

del seems to have some memory which puzzles me. See the following:

In [1]: import math

In [2]: math.cos(0)
Out[2]: 1.0

In [3]: del math.cos

In [4]: math.cos(0)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-4-9cdcc157d079> in <module>()
----> 1 math.cos(0)

AttributeError: module 'math' has no attribute 'cos'

Fine. Let's see what happens if we delete the whole math package:

In [5]: del math

In [6]: math.cos(0)
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-6-9cdcc157d079> in <module>()
----> 1 math.cos(0)

NameError: name 'math' is not defined

So now math itself is gone, as expected.

Now let's import math again:

In [7]: import math

In [8]: math.cos(0)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-8-9cdcc157d079> in <module>()
----> 1 math.cos(0)

AttributeError: module 'math' has no attribute 'cos'

So somehow interactive python remembers that math.cos was deleted specifically even after we deleted the whole math package and imported it again.

Where does python keeps this knowledge? Can we access it? Can we change it?

Sanjay

I don't why he need it, but it is a good way to understand how it work. – Swart

@LightnessRacesinOrbit Because they were previously asking how to get the math.cos function back after deleting it. – Goda

@Graipher: My question stands. Why are they deleting things from the standard library at all? – Frampton

1) That you do it with a standard library does not implies that you may be interested in non standard libraries. – Swart

2) It makes you understand how the library mechanism works, which could have implications in, let's say, time elapsed, use of resources, hard drive access, static variables in library, etc. – Swart

3) It makes you learn how the interpreter deals with some issues, and transpolate it to another similar problems, or even make your once interpreter, or work into it to make it better – Swart

4) Curiosity. It's the key that open the doors to knowledge. As my father says: breaking you learn. You never know how far your curiosity may take you. My father also says: the time of doubt is the time to learn (take any doubt as an inspiration to learn something). Also, it is interesting what Isaac Asimov said about curiosity when predicting Internet. – Swart

Answer 1

I would say that the package is still seen as imported. So performing import math again just redeclares the name, but with old contents.

You could use reload to make sure your module is whole again, except that some versions of python require to remove the entry in sys.modules as well, which makes the use of reload redundant:

import math
del math.cos
del math
sys.modules.pop("math")   # remove from loaded modules
import math
print(math.cos(0))  # 1.0

(this difference between various python versions, reload and import are discussed in a follow-up question: Should importlib.reload restore a deleted attribute in Python 3.6?)

Hutt

I'm afraid reload(math) does not re-introduce the math.cos function (at least on my machine). I have a separate question on this topic: #48808956 – Sanjay

yes, I edited to show a version which works without reload. – Distorted

Answer 2

A package is only read from disk once and then stored in memory as mutable singleton. The second time you import it you get the exact same singleton you have previously imported, and it's still missing its cos. del math merely deletes the local name for it, it doesn't "unimport" the package from Python overall.

Kalliekallista

To add one more detail -- the second import math is going to look in sys.modules to find the loaded (and still mutated) module, and skip any attempt to load again from disk. You could try to delete it from sys.modules to force a reload, or use importlib, etc. – Porta

As I said in a comment below and elsewhere, reload did not helped me in python 3.6.4. Is this a reload bug or what? #48808956 – Sanjay

@Sanjay I do not understand your comment. If you do import math then del math.cos then reload(math), then math.cos is once again available, as reload re-executes the module initialization and re-populates the module's dictionary in sys.modules. One quirk about reload is that it will retain extra attributes. So if you manually created math.foo, then when you reload(math), math.foo will remain. – Porta

@Porta reload does not work in this case on my python 3.6.4. See the link in the comment above. – Sanjay

@Sanjay Follow-up question linked here to look into the odd reload behavior in Python 3.6. – Porta

Answer 3

del math does not delete the package at all, it just deletes the local name math in the current module.

Like any other object, if any other references to the math module exist anywhere, then it's kept in memory.

And in particular, sys.modules is always a dictionary of all loaded modules, so at least there's always a reference there.

Edit: But there's a way to actually reload a module, imp.reload.

Unfortunately I can't get it to work for this case, reload needs the random module (probably to create some part of the compiled Python file), the random module needs math.cos, and it's gone. Even with importing random first there is no error, but math.cos doesn't reappear; I don't know why, maybe because it's a builtin module.

Algicide