Proper way to reload a python module from the console

Question

I'm debugging from the python console and would like to reload a module every time I make a change so I don't have to exit the console and re-enter it. I'm doing:

>>> from project.model.user import *
>>> reload(user)

but I receive:

>>>NameError: name 'user' is not defined

What is the proper way to reload the entire user class? Is there a better way to do this, perhaps auto-updating while debugging?

Thanks.

Haphazard

Answer 1

As asked, the best you can do is

>>> from project.models.user import *
>>> import project # get module reference for reload
>>> reload(project.models.user) # reload step 1
>>> from project.models.user import * # reload step 2

it would be better and cleaner if you used the user module directly, rather than doing import * (which is almost never the right way to do it). Then it would just be

>>> from project.models import user
>>> reload(user)

This would do what you want. But, it's not very nice. If you really need to reload modules so often, I've got to ask: why?

My suspicion (backed up by previous experience with people asking similar questions) is that you're testing your module. There are lots of ways to test a module out, and doing it by hand in the interactive interpreter is among the worst ways. Save one of your sessions to a file and use doctest, for a quick fix. Alternatively, write it out as a program and use python -i. The only really great solution, though, is using the unittest module.

If that's not it, hopefully it's something better, not worse. There's really no good use of reload (in fact, it's removed in 3.x). It doesn't work effectively-- you might reload a module but leave leftovers from previous versions. It doesn't even work on all kinds of modules-- extension modules will not reload properly, or sometimes even break horribly, when reloaded.

The context of using it in the interactive interpreter doesn't leave a lot of choices as to what you are doing, and what the real best solution would be. Outside it, sometimes people used reload() to implement plugins etc. This is dangerous at best, and can frequently be done differently using either exec (ah the evil territory we find ourselves in), or a segregated process.

Osy

What I'm doing to find this question, is I'm using the console to explore a data structure imported from JSON. I want to use "reload" to keep some functions in my text editor that traverse the structure in different ways, edit them, then rerun them, without having to quit out of the console. Still, if Python can't support that cleanly, for implementation reasons such as this inability to remove "leftovers", I guess I must be wrong to want to do it ;-p +1 for the answer that lets me do it anyway, though. – Bundy

+1 for showing me that I should, really really, be writing tests instead of doing things in interpreter by hand, even for fine tuning. This would also be useful for future non-regression testing. – Curtcurtail

You'll rarely see me arguing against unit tests but there are definitely scenarios when you're prototyping where this would be valuable. Not disagreeing with the overall sentiment, but 'one of the worst ways' is a pretty severe value judgement. – Locklear

Naively with medwards. I found my way here b/c using reload inside of python debugger. When debugging an editable package I was hoping to modify and reload a module and call the routine in question, rather than exit python and start from scratch each time. Why is that not a valid and use of reload()? Scenario has nothing to do with unit tests. – Ludivinaludlew

3.x keeps reload. It just moved it out of built-in functions: #961662 – Briscoe

Alright. After grumpily coming back to this reload nonsense, I've decided to go ahead and finally learn how to debug properly. Thanks for the motivation, and for anyone else having the same trouble, I'm using this website as guidelines for now: codeburst.io/how-i-use-python-debugger-to-fix-code-279f11f75866 – Sympathetic

Answer 2

For python3.4+, reload has been moved to the importlib module. you can use importlib.reload(). You can refer to this post.

>>> import importlib
>>> import project # get module reference for reload
>>> importlib.reload(project.models.user) # reload step 1
>>> from project.models.user import * # reload step 2

For python3 versions before 3.4, the module to import is imp (instead of importlib)

Alcaeus

The imp module is deprecated in favor of importlib since Python 3.4. Ref: docs.python.org/3/library/imp.html – Depositary

This should have been an update to the accepted answer. It was clearly copied from it, the comments are still there... – Natural

Answer 3

IPython can reload modules before executing every new line:

%load_ext autoreload
%autoreload 2

Where %autoreload 2reloads "all modules (except those excluded by %aimport) every time before executing the Python code typed."

See the docs:

• https://ipython.org/ipython-doc/3/config/extensions/autoreload.html

Illjudged

Answer 4

You can't use reload() in a effective way.

Python does not provide an effective support for reloading or unloading of previously imported modules; module references makes it impractical to reload a module because references could exist in many places of your program.

Python 3 has removed reload() feature entirely.

Giant

It's not module references that do it: reload() actually mutates the module object. It's the references to functions and classes inside the module that break things (in particular, the references generated by from foo import *). – Osy

Yes i agree, and you could have other program modules that has imported the module you want to unload(). – Giant

Answer 5

Unfortunately you've got to use:

>>> from project.model import user
>>> reload(user)

I don't know off the top of my head of something which will automatically reload modules at the interactive prompt… But I don't see any reason one shouldn't exist (in fact, it wouldn't be too hard to implement, either…)

Now, you could do something like this:

from types import ModuleType
import sys
_reload_builtin = reload
def reload(thing):
    if isinstance(thing, ModuleType):
        _reload_builtin(thing)
    elif hasattr(thing, '__module__') and thing.__module__:
        module = sys.modules[thing.__module__]
        _reload_builtin(module)
    else:
        raise TypeError, "reload() argument must be a module or have an __module__"

Groundwork

Answer 6

You could also try twisted.python.rebuild.rebuild.

Barbrabarbuda

Answer 7

    from test_reload import add_test

where test_reload is a module, and add_test is a function if you changed the function add_test, of course you need to reload this function. then you can do this:

    import imp
    imp.reload(test_reload)
    from test_reload import add_test

this will refresh the function add_test.

so you need to add

imp.reload(test_reload)
from test_reload import add_test  --add this line in your code

Wooded

It would be better if you add some information to explain your answer/code. – Johnsonjohnsonese

Answer 8

As of Python 3.4 you can use importlib.reload(module)

>>> from importlib import reload
>>> from project.model import user
>>> reload(user)

Ultraviolet