Should python imports take this long?

Question

For the following command

%time python test.py

on this script, test.py

import numpy as np
from math import * 
import matplotlib.pyplot as plt
import matplotlib.cm as cm
from matplotlib.colors import LogNorm
from scipy import stats

I get the output:

real    0m1.933s
user    0m1.322s
sys     0m0.282s

Is there something wrong? Or is this how long imports should take?

Caravel

Note that starting from Python 3.7 you can use the "-X importtime" option to measure the timing of every import docs.python.org/3/using/… – Quintic

Answer 1

Why imports can be slow

Import statements execute module code on import. If this is just class/function definitions it is usually very fast; some packages/modules run expensive code on startup. Here is a short example:

my_module1.py:

def func1():
    print("ran func1")

my_module2.py:

import time

def import_func():
    print("running some import-time code...")
    time.sleep(1)
    print("import startup complete")

# expensive code that executes on import
import_func()

def func2():
    print("ran func2")

Working with slow imports

Slow imports can be cumbersome if you run them often, such as rerunning a script frequently to update with changes (the reason the question was origially asked). Thankfully, ipython now has autoreload that detects code changes and reloads them automatically the next time they are called. This can be manually configured if it doesn't work out of the box. In the example above, autoreload would have behavior like the following:

import my_module1

my_module1.func1()
# prints: "ran func1"

# update func1 on-file as follows: 
# def func1():
#    print("ran modified func1")

my_module1.func1()

# prints: "ran modified func1" if autoreload is correctly configured.

Caravel

Answer 2

Some modules initialize when you use them, while others initialize everything once you start it up. Matplotlib is one of those modules.

Since matplotlib is a huge package that includes a whole lot of functionality, I'm not surprised that it takes this long, although it can get annoying.

So, in answer to your question, yes to some.

If you want a "solution" to your problem, you might want to import matplotlib only when you're going to use it, or have a loading screen / print at the beginning of your program.

Hulburt

that's helpful. For rapid debugging, is there a way to initialize once, such that its not needed when you run again? – Caravel

Not really unless you want to keep a running module consistently. If you want to, you can import submodules rather than a whole module. This of course only works if the module initializing everything at startup. – Administration

I guess one way to do it is to make a separate script that acts as a quick and dirty command prompt that imports matplot lib, and then runs a script whenever you hit Enter... But even then, I'm not sure if the script you want to debug will detect the added modules... – Hulburt

It's worth mentioning that Jupyter Notebooks serve the purpose of not needing to re-import modules if you are debugging code. You only need to execute the import cell once, and any following cells with errors in them can be rerun without re-importing modules. – Hills