Use PyTest within a Python project using the "src" layout

Question

I'm trying out the one of the recommended python package layouts with the src directory.

My problem is when I run pytest from the command line my tests are not finding my python package. I tried running from the top level of this directory and within the tests directory but still getting ModuleNotFoundError exception. I'm running python 3.5 with pytest-3.5.0.

What is the recommend way to execute pytest from this type of python package layout?

Cherri

Answer 1

If you're using py.test to run the tests, it won't find the application as you must have not installed it. You can use python -m pytest to run your tests, as python will auto-add your current working directory to your PATH.

Hence, using python -m pytest .. would work if you're doing it in the src directory

The structure that I normally use is:

root
├── packagename
│   ├── __init__.py
│   └── ...
└── tests
    └── ...    

This way when I simply run python -m pytest in the root directory it works fine. Read more at: https://mcmap.net/q/86064/-path-issue-with-pytest-39-importerror-no-module-named-39

Second option, if you still want to run this with py.test you can use:

PYTHONPATH=src/ py.test

and run it from the root. This simply adds the src folder to your PYTHONPATH and now your tests will know where to find the packagename package.

The third option is to use pip with -e to install your package in the editable mode. This creates a link to the packagename folder and installs it with pip - so any modification done in packagename is immediately reflected in pip's installation (as it is a link). The issue with this is that you need to redo the installation if you move the repo and need to keep track of which one is installed if you have more than 1 clones/packages.

Plebiscite

The whole idea of putting the package into src/ is to prevent it from being imported. You want to test the installed package, not the development folder. – Legislative

One reason for using src is to isolate the code for deployment management. It may not even be a package. To allow running tests against the repo code you can add the root directory to PYTHONPATH and this can be done in the .venv/bin/activate script to make it seamless. – Mauve

@NicoSchlömer Your comment states something fundamentally important. Could you please point to some reading? – Depressomotor

Answer 2

Your layout looks OK and is the today recommended.

But I assume your problem are the import in your test_*.py files. You shouldn't take care about from where to import your package in you unittests; just import them.

Install in Development Mode via --editable

How could this be done? You have to "install" your package in Developement Mode. Use the --editable option of pip. In that case not a real package is build and installed but only symlinks are used to expose your package folder (the source in developer version) to the operating system as it would be a real release package.

Now your unittest never need to care about where the package is installed and how to import it. Just import it because the package is known to the system.

Note: This is the today recommended way. Other "solutions" hacking around with manipulating sys.path or environment variables like PYTHONPATH are just from Python's early days and should be avoided today.

Austen

Answer 3

Just my two cents:

Don't use py.test, use pytest. The py.test is an old and deprecated command.

Notice Windows users; you may or may not have PYTHONPATH defined on your system. In such a case, I used the following commands to run tests:

set PYTHONPATH=src
pytest

To examine the PYTHONPATH:

echo %PYTHONPATH%
src

And the directory structure on containing src folder:

setup.py
src
    utils
        algo.py
        srel.py
        __init__.py
tests
    algo_test.py
    srel_test.py
    __init__.py

Finally, the documentation says that you might omit the __init__.py files. Removing them worked for me in this case.

Protomorphic

Importing src violates the idea of the "src" layout. – Austen

Answer 4

I think the idea behind the src layout is to isolate the package code notably from the tests. The best way I see here is to install your package and test it as it will be delivered to future users.

Much more suitable for development. No need for hacks like import src or to modify the PYTHONPATH.

• Create a virtual environment
• Build your package
• Install your package manually
• Install your test dependencies (possibly different from the package dependencies)
• Run pytest in the main directory

It works smoothly.

Vasilek

Answer 5

A much easier solution (available since pytest>=7) is to use the pytest.ini file to add the src directory to sys.path:

pytest.ini

[pytest]
pythonpath = src

Then you can run pytest from your root directory. See https://stackoverflow.com/a/50156706 for a more elaborate answer.

Demp