poetry can't find version of dependency even though it exists

Question

When bumping my python version from 3.7 to 3.8 in poetry, reinstalling all the dependencies fail with a version of the following:

ERROR: No matching distribution found for...

The distribution for that version is available at pypa, and is often the most recent version.

Simply removing the offending package doesn't fix the issue, as poetry will likely fail with other packages. After some investigation, it appears that somehow poetry isn't using pip3 to install underneath, but is instead using pip2.7.

Indeed this is supported by a deprecation alert, and the error is always reproducible if I attempt to install the same version with pip (globally or otherwise) and not pip3.

This issue is frustrating and deleting the venv alone doesn't seem to help. How can I resolve this dependency issue that shouldn't exist in the first place?

Wainscoting

Answer 1

There are two issues here which feed into each other.

1. poetry seems to consistently botch the upgrade of a venv when you modify the python versions. According to finswimmer, the upgrade should create a new virtual env for the new python version, however this process can fail when poetry uses the wrong pip version or loses track of which virtual env it's using.

2. poetry uses whatever pip is no questions asked - with no way to override and force usage of pip3.

Here are the distilled steps I used to solve this issue

1. delete the virtual env ( sometimes poetry loses track of the venv/thinks it's already activated. Best to clear the slate )

rm -rf `poetry env list --full-path`

2. create a new virtual env ( the command should fail, but the venv will be created )

poetry install

3. manually activate the virtual env

source "$(poetry env list --full-path | tail -1 | sed 's/.\{12\}$//')/bin/activate"

4. poetry install within the virtual env ( this ensures poetry is using the correct version of pip )

poetry install

Wainscoting

If the systems python is changed, poetry will always create a new venv if not already one for the minor version exists or you use in-project venv's. That's an intended behavior. The pip version inside your venv is the one provide by your python installation (this is not necessarily the same as your current system has). But if you get pip2 when you expect pip3 I guess there is something wrong on your system/configuration. What do you mean by "create a new virtual env ( this should fail )"? – Chronopher

Re: What do you mean: I amended the comment to be a little bit more clear – Wainscoting

Re: I guess there is something wrong on your system/configuration: Is poetry unable to realize which version of pip it should use ( depending on the version of python set )? – Wainscoting

Poetry 1.1.11 lists the output of poetry env list --full-path with (Activated) amended, making this solution not work as-is. – Ishmaelite

Fixed and updated – Wainscoting

If you make a mistake in step 3 (like me not changing "bin" to "Scripts"), when you amend it and try again the problem @DiningPhilosopher had will then occur. – Conchoidal

Answer 2

Sometimes, the import <NAME> isn't the same as pip install <NAME> / poetry add <NAME>.

Example:

import icd10

Install / Poetry:

pip install icd10-cm
poetry add icd10-cm

It's convention that the import name is the same as the install name - which isn't always the case.

This is because install names must be unique, but import names don't actually have to be, since we can add an alias:

import icd10 as foo

Source

Waldenburg

Answer 3

It can also be a matter of cache clearing sometimes. I had the same problem and this worked for me: poetry cache clear pypi --all + try again

Magistracy