How to import a module from a different folder?

Question

I have a project which I want to structure like this:

myproject
├── api
│   ├── __init__.py
│   └── api.py
├── backend
│   ├── __init__.py
│   └── backend.py
├── models
│   ├── __init__.py
│   └── some_model.py
└── __init__.py

Now, I want to import the module some_model.py in both api.py and backend.py. How do I properly do this?

I tried:

from models import some_model

but that fails with ModuleNotFoundError: No module named 'models'.

I also tried:

from ..models import some_model

which gave me ValueError: attempted relative import beyond top-level package.

What am I doing wrong here? How can I import a file from a different directory, which is not a subdirectory?

Valadez

Answer 1

Firstly, this import statement:

from models import some_model

should be namespaced:

# in myproject/backend/backend.py or myproject/api/api.py
from myproject.models import some_model

Then you will need to get the directory which contains myproject, let's call this /path/to/parent, into the sys.path list. You can do this temporarily by setting an environment variable:

export PYTHONPATH=/path/to/parent

Or, preferably, you can do it by writing a setup.py file and installing your package. Follow the PyPA packaging guide. After you have written your setup.py file, from within the same directory, execute this to setup the correct entries in sys.path:

pip install --editable .

Calculating

This will work, but you should also be able to do a relative import from within the myproject package, which OP has tried. What would cause that not to work? – Gimel

That has been covered a billion times over at Relative imports for the billionth time, so I won't go into it again. – Calculating

you think OP is running api.py as a script? that would explain it, yes – Gimel

I guess I'll dive into the PyPA packaging guide then =) – Valadez

I created a package and installed it using pip install --editable as suggested. Works like a charm, thank you! – Valadez

This answer really helped me. Although I’ve seen similar answers on many other similar questions, I couldn’t believe you have to install the package you want to develop itself. That is – at least for me – highly contra intuitive. Your answer clarified it. – Gomel

After doing pip install --editable . if we made a change on any file under the imported folder, should we do pip install --editable . again to fetch the new updates? – Alvita

@Alvita it should not be necessary. The whole point of the —editable installation is that you are able to edit the installed code directly. – Calculating

Answer 2

Unfortunately, Python will only find your file if your file is in the systems path. But fear not! There is a way around this!

Using python's sys module, we can add a directory to the path while Python is running just for the current run of the program. This will not affect any other Python programs run later.

You can do this by:

import sys
import os
sys.path.insert(0, os.path.abspath('relative/path/to/application/folder'))
import [file]

It is important to import sys and set the directory path before you import the file.

If your library is NOT close to the current directory, an absolute path could be simpler:

import sys
sys.path.insert(0, '/absolute/path/to/application/folder'))
import [file]

Yes, this is a hack. But it can be helpful as a short-term work-around.

Singlebreasted

Changing the path from within code, though possible, is messy and bad - it makes the test setup more difficult and makes the import ordering become significant. – Calculating

Ah, when will python solve this problem, why python 3.10 still not solve this problem, is this very difficult ? – Batting

Yes, this is a hack. – Sequential

It would be useful to convert a relative path to an absolute one. import os and os.path.abspath("mydir/myfile.txt") will accomplish this. – Sequential

@WadeWang This is fundamentally a system configuration issue. Package managers like pip should help. Or package development managers -- but I don't have much experience with those. This is a useful hack. That doesn't mean I recommend it for any code that we intend to have last! – Sequential

@JosiahYoder IMO, suggesting this at all does more harm than good. If you are going to hack the system path, it makes way more sense to do it by setting the PYTHONPATH environment variable before the interpreter is even invoked – Steffens

@Steffens Yes, if you can configure your app like a 12-factor app, the system environment variable is the way to go. In some research contexts where you are just exploring, this is still a useful hack. – Sequential

Answer 3

I would lay out two approaches:

Simply import some_model via absolute importing:

from myproject.models import some_model

Note that the myproject should be treated as an module (i.e. having __init__.py)

---

Or

You can add the previous path to the sys.path which I use in such parallel level modules:

import sys
sys.path.append('../')

from models import some_model

Bowerbird

Answer 4

Due to the file not being in your systems path, you will need to append the parent directory using import sys, like so.

import sys
sys.path.append('myproject')
from models import some_model

This is the simplest way to do it, because the module you want to access is so close to your parent directory. So now your systems path will be in myproject.

Hecate

this approach is already detailed in other answers, and it is also definitely not what you should be doing in almost all cases. – Steffens

Oh good to know. What way would you suggest for most other cases? Eager to learn from you. – Hecate