Why pytest always says " ImportError: attempted relative import with no known parent package"

Question

I'm new to pytest and python. My project structure is :

projroot/
|-src/
  |-a.py
|-test/
  |-test_a.py

and test_a.py is:

from ..src import a
def test_a():
  pass

Then run pytestunder projroot:

projroot>pytest

And always there is the error info: E ImportError: attempted relative import with no known parent package. Python verison: 3.8 on windows 10 x64. I read a lot articles and blogs and official suggestions, but still failed to figure out. Help me, please!

Actually

You need to add (possibly empty) __init__.py files so that Python knows these are packages. At least one in src, perhaps also in test and possibly in projroot too. That should solve your issue. – Threesquare

Thank you, filbranden and napuzba! Solved! I found __init__.pyfiles are essential in test and projroot, but not in src. – Actually

Answer 1

I found I must add __init__.py in test and projroot. __init__.py in src is not necessary. And pytest could run correctly in 3 folders:

# parent folder of project root
C:\>pytest projroot

# project root
C:\>cd projroot
C:\projroot>pytest

# test folder
C:\projroot>cd test
C:\projroot\test>pytest

Actually

Answer 2

Adding __init__.py to project root actually messed up my pytest discovery and also pylint (causing Unable to import 'angles' pylint(import-error)). I had to remove it from project root and only add it inside my tests folder, which solved both my pytest discovery and pylint warning.

With __init__.py in project root, i'll have ModuleNotFoundError: No module named 'angles'

Below is my folder structure with both pytest discovery and pylint (without warning) working :

powerfulpython
├── angles.py
└── tests
    ├── __init__.py
    └── test_angles.py

In angles.py I have a Class Angle which I try to absolute import using from angles import Angle inside test_angles.py.

Before I found out that the issue was having an unnecessary __init__.py in project root. I also tried

1. editing python.testing.cwd
2. setting '--rootdir' of pytest
3. Downgrading Python extension to 2020.9.114305 (following https://github.com/microsoft/vscode-python/issues/14579)

None of them worked.

Before i deleted the __init__.py in project root, when testing pytest, doing python -m pytest worked because the project root is prepended to sys.path automatically so the absolute import was working correctly from root and the import statement can find angles under root. Doing pytest (without python -m) fails because the project root is not prepended to sys.path. The latter can be temporarily solved by adding the root directory manually to $PYTHONPATH. However this is an unsustainable solution as I don't want to add every single new project i want to test as a PYTHONPATH edit to my ~/.zshrc.

Another hack I tried which fixed both pytest and test discovery was inserting sys.path.insert(0,'/Users/hanqi/Documents/Python/powerfulpython') in the code but this is way too ugly. I didn't want to add lines to code just for testing then have to delete them later.

After all these, I deleted __init__.py in root and all's good. No need for any of the above list of 3 items.

Many solutions I went through deal with finding the tests folder, but that was not my problem. My tests were found correctly in their folders, just that the from angles import Angle was breaking which causes test discovery to fail.

Investigating how does __init__.py in project root break things:

I printed sys.path and realized with it, ~/Documents/Python got prepended to sys.path and without it, ~/Documents/Python/powerfulpython got prepended. The latter path is exactly my project root.
I'm guessing this has something to do with https://docs.pytest.org/en/6.2.x/pythonpath.html (--import-mode=prepend (default)). I was expecting to see the __init__.py inside tests to cause ~/Documents/Python/powerfulpython to be prepended and the __init__.py in project root cause ~/Documents/Python to be prepended, but strangely i only saw the latter when I left both __init__.py inside.

I also tried deleting the __init__.py inside tests (but let __init__.py in root stay), and see ~Documents/Python/powerfulpython/tests prepended to sys.path. This same line is also prepended if I delete both __init__.py in root and tests, not sure why.

Edit: The above behaviour is explained at https://docs.pytest.org/en/6.2.x/pythonpath.html

It will then search upwards until it can find the last folder which still contains an __init__.py file in order to find the package root

After further repeating my above experiments running using python -m pytest -s (-s so successful test runs don't swallow my print(sys.path)) instead of just pytest, I confirm that there are 2 actors prepending to sys.path.

1. python -m (what it prepends depends on from which directory the command is run)
2. pytest (what it prepends depends on where are __init__.py inserted (explained in pytest docs linked above)

Both have the effect of editing sys.path, helping to make failing imports possible

Jacques Yang above needed 2 __init__.py to put the containing folder of his projroot into sys.path so projroot.test appears as the value of __package__ in his test_a.py and __name__ would appear as projroot.test.test_a, and the 2 dots in this module name (__name__) allow the two dot relative import of from ..src import a. He could also have used an absolute import of from projroot.src import a.

If he didn't add the __init__.py in projroot, the __package__ would become just test and __name__ would be test.test_a, which would only allow an upwards relative step of at most 1 step. (This stepping concept is also explained by BrenBarn at Relative imports for the billionth time).

When there is __init__.py in test but not projroot, that would have been ValueError: attempted relative import beyond top-level package, because __package__ is test and __name__ is test.test_a allowing only 1 step back up but 2 steps back was coded. The question in the title occurs when there isn't even __init__.py in test, so __package__ is completely blank and __name__ is only the test_a file itself)

I was thinking why did OP need 2 __init__.py while I only needed 1, and 2 actually harms my attempt. Then I realized OP was doing relative import and I was doing absolute import with from angles import Angle. If i followed OP's pattern and did relative import from ..angles import Angle, then I also would have needed to add __init__.py under powerfulpython, otherwise i'll get the same errors.

So why does doing absolute import break with the extra __init__.py in ~/Documents/Python/powerfulpython? Because the __init__.py causes pytest to step up even higher in the hierarchy and prepend ~/Documents/Python to sys.path, but my absolute import needed to have ~/Documents/Python/powerfulpython (this can be resolved with running using python -m pytest as explained in the 2 actors of prepending section above, but feels somewhat unnatural compared to simply pytest)

I have a 30-min demo video on import concepts at https://towardsdatascience.com/taming-the-python-import-system-fbee2bf0a1e4 that could help people understand __package__ and __name__.

Sisneros

Answer 3

By including a setup.py file in root directory and init.py in every folder worked for me and for importing a package inside any file just give that package location from parent directory

setup.py

from setuptools import setup, find_packages

setup(name="PACKAGENAME", packages=find_packages())

projroot/
|-__init__.py
|-setup.py
|-src/
  |-a.py
  |-__init__.py
|-test/
  |-test_a.py
  |-__init__.py

from src import a
def test_a():
  pass

Reference https://docs.pytest.org/en/6.2.x/goodpractices.html

Varmint