Python import from parent directory and keep flake8 happy

Question

This import works fine, but feels dirty in a few ways. Mainly that it uses a specific number in the slice* to get the parent path, and that it annoys the flake8 linter.

import os
import sys
sys.path.append(os.path.dirname(__file__)[:-5])
from codeHelpers import completion_message

It's in a file system that looks a bit like this:

parent_folder
    __init__.py
    codeHelpers.py

    child_folder
        this_file.py

(child_folder is actually called week1, hence the 5 in the slice)

This question is extremely similar to Python import from parent directory, but in that case the discussion focused on whether or not it was good to run tests from the end point. In my case, I have a series of directories that have code that uses helpers that live in the parent.

Context: each directory is a set of weekly exercises, so I'd like to keep them as simple as possible.

Is there a cleaner, more pythonic way to do this import?

@cco solved the number problem, but it's still upsetting the linter.

Goober

you have a lot of answers that solve your problem, if you could please mark one of them as answered I'm sure they would appreciate it. – Checkbook

Answer 1

First since you haven't been specific about which lint error you are getting, I am going to assume it's because you have an import after your sys.path.append.

The cleanest way to do it is with relative or absolute imports.

Using absolute imports:

from parent_path.codeHelpers import completion_message

Using relative imports:

from ..codeHelpers import completion_message

For the simple example listed in the original question this should be all that's required. It's simple, it's pythonic, it's reliable, and it fixes the lint issue.

You may find yourself in a situation where the above does not work for you and sys.path manipulation is still required. A drawback is that your IDE will likely not be able to resolve imports to modules from the new path causing issues such as automatic code completion not working and flagging the imports as errors, even though the code will run properly.

If you find you still need to use sys.path and want to avoid lint errors for this type of situation create a new module and do the sys.path manipulation in it instead. Then make sure that you import your new module before any modules that require the modified sys.path.

For example:

local_imports.py

"""Add a path to sys.path for imports."""

import os
import sys

# Get the current directory
current_path = os.path.dirname(__file__)

# Get the parent directory
parent_path = os.path.dirname(current_path)

# Add the parent directory to sys.path
sys.path.append(parent_path)

Then in the target file:

import local_imports  # now using modified sys.path
from codeHelpers import completion_message

The drawback to this is it requires you to include local_imports.py in each child_folder and if the folder structure changes, you would have to modify each one local_imports file.

Where this pattern is really useful is when you need to include external libraries in your package (for example in a libs folder) without requiring the user to install the libs themselves.

If you are using this pattern for a libs folder, you may want to make sure your included libraries are preferred over the installed libraries.

To do so, change

sys.path.append(custom_path)

to

sys.path.insert(1, custom_path)

This will make your custom path the second place the python interpreter will check (the first will still be '' which is the local directory).

Checkbook

Answer 2

You can import from a module a level up in a package by using ... In this_file.py:

from ..codeHelpers import completion_message

Had you wanted to go more levels up just keep adding dots...

While I'm here, just be aware that from ..codeHelpers is a relative import, and you should always use them when importing something in the same package. from codeHelpers is an absolute import, which are ambiguous in Python 2 (should it import from in the package or from the unfortunately named codeHelpers module you have installed on your system?), and in Python 3 actually forbidden as a way to import from within the same module (i.e. they are always absolute). You can read the ancient PEP 328 for an explanation of the difficulties.

Drisko

It is a bit unfortunately named, code is the name of the degree, computational design. That's the logic, but I can see how it looks like I've called my helpers file programmingStuff – Goober

I've tried from ..codeHelpers but I get a ValueError: Attempted relative import in non-package error. Does that imply a problem elsewhere or does it mean that I need to call this file from a different file rather than running it directly? – Goober

So I take it that means that you're running this_file.py directly. Normally I'd advise against mixing module and script code. But if you want to make a package also a script the way to do it is using the __main__.py mechanism. Basically you'd rename this_file.py to __main__.py file in child_folder and execute it by running python -m parent_folder.child_folder, making sure you have __init__.py files in both parent_folder and child_folder (if using python 2; in python 3 it's unnecessary). – Drisko

Answer 3

It might be easier to use absolute import paths, like the following:

from parent_folder.code_helpers import completion_message

But this would require you to make sure that the PYTHONPATH environment variable is set such that it can see the highest root directory (parent_folder in this case, I think). For instance,

PYTHONPATH=. python parent_directory/child_directory/this_file.py
# here the '.' current directory would contain parent_directory

Make sure to add an __init__.py to the child_directory as well.

Superstar

Answer 4

You can remove the assumption about the length of the final directory name by applying os.path.dirname twice.

e.g. instead of os.path.dirname(__file__)[:-5], use os.path.dirname(os.path.dirname(__file__))

Sodomite

Answer 5

Either way you have to hack around. If you main goal avoid flakes warnings

• add a noqa comment

• exec(open("../codeHelpers.py").read(), globals())

• you can pass a filename with interpreter option -c (should not bother flakes8)

Aarika