How to accomplish relative import in Python

Question

Consider:

stuff/
    __init__.py
    mylib.py
    Foo/
        __init__.py
        main.py
        foo/
            __init__.py
            script.py

script.py wants to import mylib.py.

This is just an example, but really I just want to do a relative import of a module in a parent directory. I've tried various things and get this error...

Attempted relative import beyond toplevel package

I read somewhere that the script from where the program starts shouldn't reside in the package, and I tried modifying the structure for that like so...

stuff/
    mylib.py
    foo.py // Equivalent of main.py in above
    foo/
        __init__.py
        script.py

But I got the same error.

How can I accomplish this? Is this even an adequate approach?

In Python 2.

Adverb

Answer 1

After fiddling with it a bit more, I realized how to set it up, and for the sake of specificity I won't use foo bar names. My project directory is set up as...

tools/
    core/
        object_editor/
            # files that need to use ntlib.py
            editor.py # see example at bottom
            __init__.py
        state_editor/
            # files that need to use ntlib.py
            __init__.py
        ntlib.py
        __init__.py # core is the top level package
    LICENSE
    state_editor.py # equivalent to main.py for the state editor
    object_editor.py # equivalent to main.py for the object editor

A line in object_editor.py looks like...

from core.object_editor import editor

A line in editor.py looks like...

from .. import ntlib

or alternatively

from core import ntlib

The key is that in the example I gave in the question, the "main" script was being run from within the package. Once I moved it out, created a specific package (core), and moved the library I wanted the editors to share (ntlib) into that package, everything was hunky-dory.

Adverb

You've got it. What's going on is that you can't use relative imports from the script that you're running from the command line, so that should be at the top level of organisation, referring to things below it. – Sharpen

Why the "main" script was being run from within the package cause the problems ? – Favourable

Answer 2

Though as long as "stuff" is not in your Python PATH, you haven't got any choice than adding the path.

If you know the level of your script.py from stuff, you can do, for example:

import sys
import os
sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..'))

Koh

Is this documented somewhere? Not that I don't believe you, but in the case of just wanting to say "hey import from 2 directories up" requires modifying the system path, I've gotta see it with my own eyes. – Adverb

This is what I usually end up doing, if only out of spite. – Hophead

I wouldn't say its exactly documented like this. though I used this multiple times myself. It will just modify the system path for your running python script, not globally in the system anyway. I think you should not mind. :P – Koh

Works great, particularly for unit tests, where you just cannot change the layout. – Dieball

env PYTHONPATH=.. python myfile.py Is another way as well – Confine

The problem with this is that most (all?) python IDEs won't always honour this and will therefore flag the imports as unresolvable errors, breaking indexing and almost everything useful about the IDE. In some (e.g. PyCharm) this can be resolved somewhat by manually specifying the correct 'root' for the absolute imports in the project settings. However I haven't been able to get this to work with relative imports at all, so far. – Flax

In the end you should not really have to do this though. This answer is already quite old, but I believe that I never had to do this since then. It is all about good dependency management and how you structure your project. – Koh

Answer 3

I'm running Python 3.4.2 on Windows 7 and tore my hair out over this.

When running either of these:

python -m unittest python -m unittest discover

...I would get the 'Attempted relative import beyond toplevel package' error.

For me, the solution was dropping the ".." in my [test_stock.py]. The line was: from ..stock import Stock

Changed it to: from stock import Stock

.. and it works.

Folder structure:

C:\
  |
  +-- stock_alerter
             |
             +-- __init__.py
             +-- stock.py
             |
             \-- tests
                   |
                   +-- __init__.py
                   \-- test_stock.py

Karlsbad

Your solution works but I can't understand why...Plus, Pycharm is getting crazy with this and red-highlight the statement – Bocanegra

It works because it's no longer a relative import when you remove the ... Instead it's an absolute import. PyCharm will use the project's source directory to resolve it, which means you might need to manually add a directory in the project settings to help it resolve the imports. – Flax

Answer 4

From the PEP it appears that you cannot use a relative import to import a file that is not packaged.

So you would need to add a __init__.py to stuff and change your imports to something like from .mylib import *

However, the PEP seems to make no allowance to keep mylib packaged up in a module. So you might be required to change how you call your library functions.

Another alternative is to move mylib into a subpackage and import it as from .libpackage import mylib

Impervious

Answer 5

If you're on Linux or perhaps a similar Unix-like system, you can hack this with symbolic links.

stuff/
    mylib.py
    foo.py // equivalent of main.py in above
    foo/
        script.py
        mylib.py  ->  ../mylib.py
    foo2/
        script2.py
        mylib.py  ->  ../mylib.py

This is likely not a good pattern to follow.

In my case, I opted for it, because I had multiple executables dependent on the same library that needed to be put into separate directories.

Implementation of new executable tests shouldn't require the test writer to have a deep understanding of Python imports.

tests/
    common/
        commonlib.py
    test1/
        executable1.py
        executable2.py
        commonlib.py -> ../common/commonlib.py
    test2/
        executable1.py
        executable2.py
        commonlib.py -> ../common/commonlib.py

Asteria