python3.x : ModuleNotFoundError when import file from parent directory

Question

I am new to Python. This really confused me!

My directory structure is like this:

Project
   | - subpackage1
           |- a.py
   | - subpackage2
           |- b.py
   | - c.py  

When I import a.py into b.py with from subpackage1 import a, I get a ModuleNotFoundError. It seems like I cannot import a file from the parent directory.

Some solutions suggest to add an empty file __init__.py in each directory, but that does not work. As a work around, I have put the following in each subfile (i.e., a.py and b.py) to access the parent directory:

import os
import sys
sys.path.append(os.path.abspath('..'))  

I have tried to output sys.path in subfiles, it only includes the current file path and anaconda path, so I have to append .. to sys.path.

How can I solve this problem? Is there a more efficient way?

Engracia

Answer 1

The first issue you're getting is due to that from subpackage1 import a line in your b.py module.

b.py is located in your subpackage2 package, a sibling package of subpackage1. So trying to run from subpackage1 import a means the subpackage1 is within subpackage2 which is incorrect. Also as side note, in python3 you should never use implicit relative imports as it no longer supports it, so instead use explicit relative imports.

Adding an __init__.py in each folder turns them in python packages, and you can keep them empty. You want to replace the from subpackage1 import a with either an explicit relative import like from ..subpackage1 import a or an absolute import like from Project.subpackage1 import a. This will be the efficient and correct way to write your package, and you can test it by writing a script that imports Project and uses its subpackages and modules.

However, I am assuming you are running b.py as a main module to test the imports. This means you are running command lines that look like: python b.py. Running it like this gives you a module search path that does not have any of the parent paths like Project. This would lead to you keep getting ModuleNotFoundErrors even though there's nothing technically wrong with your package. This is why you need a sys.path.append(... work-around that manually appends your parent path to the module search path in order to run your b.py module as a main module. If this helps you to test your code then by all means use it, but it is fine to use the absolute and explicit relative imports because modules in a package are supposed to work that way.

Nazler

Okay, thank you very much, your answer makes me a little clearer. It means that if b.py in subpackage2 needs a parameter in a.py, and then I should add sys.path.append(...) at the head of b.py, and if I don't do that I should move a.py or subpackage1 in subpackage2, right? – Engracia

You are close! If you want to run python b.py successfully, you can use the sys.path.append(os.path.abspath('..')) line in your b.py module which manually puts the parent package path (the "Project" path in this case) in your module search path. This lets you import any module in subpackage1 in your b.py module when it's run as a main module (running the module like python b.py). However, this may give you problems when you try to run a script that imports Project. Moving a.py into subpackage1 is an option, but gets away from the heart of your original question. – Nazler

Answer 2

Suppose we have these files and directories tree:

$> tree
.
├── main.py
├── submodule1
│   ├── a.py
│   └── __init__.py
└── submodule2
    ├── b.py
    └── __init__.py

2 directories, 5 files

So, here is an example of how to do the import from a.py inti b.py and vice-versa.

a.py

try:
    # Works when we're at the top lovel and we call main.py
    from submodule1 import b
except ImportError:
    # If we're not in the top level
    # And we're trying to call the file directly
    import sys
    # add the submodules to $PATH
    # sys.path[0] is the current file's path
    sys.path.append(sys.path[0] + '/..')
    from submodule2 import b


def hello_from_a():
    print('hello from a')


if __name__ == '__main__':
    hello_from_a()
    b.hello_from_b()

b.py

try:
    from submodule1 import a
except ImportError:
    import sys
    sys.path.append(sys.path[0] + '/..')
    from submodule1 import a


def hello_from_b():
    print("hello from b")


if __name__ == '__main__':
    hello_from_b()
    a.hello_from_a()

And, main.py:

from submodule1 import a
from submodule2 import b


def main():
    print('hello from main')
    a.hello_from_a()
    b.hello_from_b()


if __name__ == '__main__':
    main()

Demo:

When we're in the top level and we're trying to call main.py

$> pwd
/home/user/modules
$> python3 main.py
hello from main
hello from a
hello from b

When w're in /modules/submodule1 level and we're trying to call a.py

$> pwd
/home/user/modules/submodule1
$> python3 a.py
hello from a
hello from b

When we're /modules/submodule2 level and we're trying to call b.py

$> pwd
/home/user/modules/submodule2
$> python3 b.py
hello from b
hello from a

Bitolj

Hi @Chiheb Nexus, thanks for your response. It means that if I want to import parent file or the file in the same level, I have to append the parent directory in the sys.path, right? – Engracia

No, it depends on which level you're. My example shows how you can run the code in each file's level. – Bitolj

Answer 3

Firstly, you need create file __init__.py in subpackage1 to declare it is a module

touch subpackage1/__init__.py

Secondly, you can try relative import in python3

# b.py
from ..subpackage1 import a

Or you can add your current directory to $PYTHONPATH

export PYTHONPATH=${PYTHONPATH}:${PWD}

Oculus

Answer 4

One way to access the subpackage is using . operator going all the way up to top package or file directory. Thus if the structure is

top_directory
|- package1
   |- subpackage1
      |- a.py
|- package2
   |- subpackage2
      |- b.py

Then you use the following

#b.py
from top_directory.package1.subpackage1 import a

statements...

The from statement must go all the way up to the top directory covering both a.py and b.py.

Decalogue

Answer 5

This SO Q&A discusses options for adding your project root to the PYTHONPATH.

1. mac/linux:
add export PYTHONPATH="${PYTHONPATH}:/path/to/Project_top_directory" to your ~/.bashrc

2. path config file:
Alternatively, you can create a path configuration file

# find directory
SITEDIR=$(python -m site --user-site)

# create if it doesn't exist
mkdir -p "$SITEDIR"

# create new .pth file with our path
echo "$HOME/foo/bar" > "$SITEDIR/somelib.pth"

Gallice