`ModuleNotFoundError: No module named` after poetry install for package

Question

I have a package configured to use poetry for build / install during a container build. I cannot understand why poetry install says it has installed my package, yet it is not found. However, poetry build followed by a pip install of the wheel works fine. Is there some requirement to specify a subpackage if the package name is the same folder name as the root folder or how do I make poetry install the full package using poetry install? I have tried various adjustments to the pyproject.toml file including setting packages.

The file structure looks like this:

.
└──  my_pkg
    ├── Dockerfile
    ├── pyproject.toml
    └── my_pkg
        ├── __init__.py
        └── app.py

The pyproject.toml file looks like this:

[tool.poetry]
name = "my_pkg"
version = "0.1.0"
description = "My Package"

[tool.poetry.dependencies]
python = ">=3.8,<3.11"
...

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

The issue is that the package is not installed after running poetry install.

%  poetry install
Installing dependencies from lock file

Package operations: 70 installs, 4 updates, 0 removals

   • Installing pyasn1 (0.4.8)
   • Installing cachetools (4.2.4)
   • Updating charset-normalizer (2.0.9 -> 2.0.8)
   • Installing frozenlist (1.2.0)
   • Installing multidict (5.2.0)
...


Installing the current project: my_pkg (0.1.0)

This results in the package not being found:

$ python
Python 3.9.9 (main, Dec  3 2021, 01:42:21) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import my_pkg
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'my_pkg'

If I try to find the actual files, I cannot find them, but pip thinks the package is installed.

$ find /usr/local -name my_pkg\*
/usr/local/lib/python3.9/site-packages/my_pkg.pth
/usr/local/lib/python3.9/site-packages/my_pkg-0.1.0.dist-info
$ ls -la /usr/local/lib/python3.9/site-packages/my_pkg-0.1.0.dist-info
total 32
drwxr-xr-x 2 root root  4096 Dec  8 15:32 .
drwxr-xr-x 1 root root 12288 Dec  8 15:32 ..
-rw-r--r-- 1 root root     6 Dec  8 15:32 INSTALLER
-rw-r--r-- 1 root root  1535 Dec  8 15:32 METADATA
-rw-r--r-- 1 root root   454 Dec  8 15:32 RECORD
$ cat /usr/local/lib/python3.9/site-packages/my_pkg.pth 
/app/my_pkg
$ pip freeze | grep my_pkg
my_pkg==0.1.0

Here, the /app/my_pkg appears to be the temporary path where the code was when doing poetry install, but this does not persist in the resulting container.

However, when using poetry build && pip install dist/my_pkg*.whl, the correct package appears to be included. All of the .py files show up under /usr/local/lib/python3.9/site-packages/my_pkg/ as well.

Building my_pkg (0.1.0)
  - Building sdist
  - Built my_pkg-0.1.0.tar.gz
  - Building wheel
  - Built my_pkg-0.1.0-py3-none-any.whl
Processing ./dist/my_pkg-0.1.0-py3-none-any.whl
...

$ python
Python 3.9.9 (main, Dec  3 2021, 01:42:21) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import my_pkg
>>> 

Mythology

Answer 1

poetry install installs the package into an venv. You have to activate it first, e.g. with poetry shell. Have a look into the docs as well.

Decameter

just ran into this using poetry for the first time and overlooked this step in their docs. Thanks! – Excerpt

no, that does not solve the problem in my case – Farming

Answer 2

If for any reason you really need poetry to not create a virtual environment when installing the packages, you can add this:

poetry config virtualenvs.create false
poetry install

Southwestwardly

Hi @Southwestwardly I did it as well. I have a containerfile to download poetry and install packages but when I go into my container and run python script. I keep getting no module unless I do poetry install once again inside container. What do I miss? – Vaden

Answer 3

I had a similar problem and this issue answer helped. Hope it helps someone else. Basically add

[tool.poetry]
[... Your stuff]
packages = [{ include = "my_pkg" }]

Here are the docs of poetry. Don't forget to add the __init__.py in your /my_pkg otherwise wont be recognized. If you are starting the module in an ENTRYPOINT of a dockerfile use dots instead of / to reference your main. For example: my_pkg.main:app

Jinajingle

this was it for me, I had added a second module to my package and forgot to update the include = "my_new_package" in the pyproject.toml. I was able to execute the new script in the poetry shell or with poetry run because I had it in the [tool.poetry.scripts] section, but when built for a homebrew formula the module wasn't found. – Kava