Python - optimize by not importing at module level?

Question

In a framework such as Django, I'd imagine that if a user lands on a page (running a view function called "some_page"), and you have 8 imports at the top of module that are irrelevant to that view, you're wasting cycles on those imports. My questions are:

1. Is it a large enough amount of resources to make an impact on a high-traffic website?
2. Is it such a bad practice to import inside of a function for this purpose that it should be avoided at said impact?

Note: This could be considered premature optimization, but I'm not interested in that argument. Let's assume, for the sake of practical theory, that this is a completed site with loads of traffic, needing to be optimized in every way possible, and the application code, as well as DB have been fully optimized by 50 PhD database admins and developers, and these imports are the only thing left.

Julius

Without profiling data, it's premature optimization. Please, please, please, get profiling data before asking hypothetical questions like this. Please get profiling data so that you can see the actual cost of these two designs. Please. Get. Data. – Silvana

For those who stumble into this question: wiki.python.org/moin/PythonSpeed/… – Julius

Please note that the assumption: "user lands on a page (running a view function called "some_page"), and you have 8 imports at the top of module that are irrelevant to that view, you're wasting cycles on those imports" Is false. The module is not reloaded from scratch for every page served. Stumbling on to this question will lead to needless confusion, since the assumption is wrong. – Silvana

Answer 1

No, don't do this. In a normal python execution environment on the web (mod_wsgi, gunicorn, etc.) when your process starts those imports will be executed, and then all subsequent requests will not re-execute the script. If you put the imports inside the functions they'll have to be processed every time the function is called.

Inrush

Answer 2

Yes, it is a bad practice to import at the function level. By using smarter imports at the top of the module, you create a one time, small cost. However, if you place an import in a function you will suffer the cost of the import each time that function is run. So, rather than import in the function, just import at the top of the module.

A few things you can do to clean up and improve your imports:

• Don't use wild imports e.g. from x import *
• Where possible, just use a normal import e.g. import x
• Try to split your code up into smaller modules that can be called separately, so that fewer imports are made

Also, placing imports at the top of the module is a matter of style. There's a reason why PEP 8 says that modules need to be imported at the top. It's far more readable and maintainable that way.

Finally, some imports at function level will cause compatibility issues in the future, as from x import * is not valid Python 3.x at function level.

Creed

interesting, I wasn't aware of that change in Python 3k. Thanks. – Julius

Answer 3

1) The answer is no. Django/Python is not like PHP. Your whole module will not be reinterpreted with each pageview like happens with PHP includes. The module will be in memory and each page view will make a simple function call to your view.

2) Yes, it will be a counter-optimization to make imports at the view level.

Baumgartner

Answer 4

1. No. Same reason as other answers.

2. Yes. Same reason as other answes.

BTW, you can also do import lazily. For example, Importing toolkit can "import" a module in top level code, but the module is not actually loaded until one of the attributes is accessed.

Circumflex

Answer 5

Sometimes following boiler-plate makes sense:

foo = None

def foorify():
    global foo
    if not foo: from xxx import foo

    foo.bar()

This makes sense when foorification is conditional on something that rarely changes, e.g. one server foorifies while another never does or if you don't want or cannot safely import foo during application startup or most tests.

Adjoin