Why is my poetry virtualenv using the system python instead of the pyenv python that I set?

Question

I've recently installed both Pyenv and Poetry and want to create a new Python 3.8 project. I've set both the global and local versions of python to 3.8.1 using the appropriate Pyenv commands (pyenv global 3.8.1 for example). When I run pyenv version in my terminal the output is 3.8.1. as expected.

Now, the problem is that when I create a new python project with Poetry (poetry new my-project), the generated pyproject.toml file creates a project with python 2.7:

[tool.poetry]
name = "my-project"
version = "0.1.0"
description = ""
authors = ["user <[email protected]>"]

[tool.poetry.dependencies]
python = "^2.7"

[tool.poetry.dev-dependencies]
pytest = "^4.6"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

It seems that Poetry defaults back to the system version of Python. How do I change this so that it uses the version installed with Pyenv?

Edit

I'm using MacOS, which comes bundled with Python 2.7. I think that might be causing some of the issues here. I've reinstalled Python 3.8 again with Pyenv, but when I hit Poetry install I get the following error:

The currently activated Python version 2.7.16 is not supported by the project (^3.8).
Trying to find and use a compatible version.

[NoCompatiblePythonVersionFound]
Poetry was unable to find a compatible version. If you have one, you can explicitly use it via the "env use" command. 

Should I create an environment explicitly for the project using Pyenv or should the project be able to access the correct Python version after running pyenv local 3.8.1.? When I do the latter, nothing changes and I still get the same errors.

Botelho

Does this issue help you / clarify the status? – Lenwood

Answer 1

Alright, I figured the problem. A little embarrassingly, I had not run pyenv shell 3.8.1 before running any of the other commands. Everything works now. Thank you all for your efforts.

Botelho

Interestingly, I needed to do this, even though I had a .python-version with the correct version in the directory. – Gastrology

If the .python-version is not being read, you might need to fix your pyenv configuration: github.com/pyenv/pyenv#basic-github-checkout – Isidraisidro

This is the solution to most of the problems I have. (And embarrassingly, I need to google this answer every time I do this!) Thank you. – Algo

@Isidraisidro that was exactly the issue in my case - I re-configured pyenv to use the updated logic and .python-version is now discovered properly by poetry: Upgrade note: The startup logic and instructions have been updated for simplicity in 2.3.0. The previous, more complicated configuration scheme for 2.0.0-2.2.5 still works. – Hyperion

Answer 2

My solution to this.

First of all see the situation with this command

poetry env list

If you have an output like this: project_name-QI_LjVaV-py3.9 (Activated)

you may want to get rid of this env.

So you do the "deactivation" first:

deactivate

and then the "remove" after :

poetry env remove project_name-QI_LjVaV-py3.9

Now the same command:

poetry env list

should return nothing.

Then you do:

which python3

and, if the version is ok, you use this same exact output of the path of python, to tell to poetry (Example):

poetry env use /usr/bin/python3

Do again

poetry env info 

to be sure that is using the version of the python you want.

You can continue with

poetry install

Kenny

Thanks! This solved the issue for me after trying many other solutions for an hour. I think removing the created env was key. – Sybyl

@Sybyl correct, the poetry documentation states: "If a virtual environment has already been created for the project under {cache-dir}/virtualenvs, setting this variable to true will not cause poetry to create or use a local virtual environment. In order for this setting to take effect for a project already in that state, you must delete the virtual environment folder located in {cache-dir}/virtualenvs." python-poetry.org/docs/configuration/#virtualenvsin-project – Barrault

Answer 3

pyproject.toml is used to define all the dependencies for your project, including the supported python version.

The line your complaining about is just saying that the versions of python supported by the project is python2.7 or greater, this is independent of what versions of python you've installed with pyenv.

python = "^2.7"

If you want to update the versions of python supported by the project you can edit the file directly and run poetry update.

---

If you want to use multiple versions of python you need to make sure poetry is using the correct dependencies for the version of python you are using. To change the specific version poetry is using you should use poetry env,

• poetry env list show the versions of python poetry can use
• poetry env use <python> switches poetry to use that version.

For instance on my machine poetry has 3 virtual environments installed and is using the one associated with python3.6:

↪ poetry env list
sipy-a9sqc5pb-py3.6 (Activated)
sipy-a9sqc5pb-py3.7
sipy-a9sqc5pb-py3.8

I'm not sure how these virtual environments with interact with the shivs used by pyenv but their docs have a section relating to it

Managing Virtual Environments

There is a pyenv plugin named pyenv-virtualenv which comes with various features to help pyenv users to manage virtual environments created by virtualenv or Anaconda. Because the activate script of those virtual environments are relying on mutating $PATH variable of user's interactive shell, it will intercept pyenv's shim style command execution hooks. We'd recommend to install pyenv-virtualenv as well if you have some plan to play with those virtual environments.

Halflength

"Poetry should manage all your python dependencies so that there is no need to use pyenv."—pyenv isn't for dependencies. It manages versions of Python itself. Does Poetry do that natively? – Foxe

Ah my mistake. The keypoint is that the pyproject.toml file is saying that your project supports python2.7 or great, it is not saying 'I want to use python3.8'. If you want to update the versions of python your project supports then update toml file directly – Halflength

I've updated my answer, hopefully it makes more sense now. – Halflength

Great answer, thanks. Although I do have a follow-up question: How do I increase the number of virtual environment options for Poetry? When I run poetry env list now, there's only one option and that's for Python 2.7. What I'd like then is to also have and option for Python 3.8 for example. – Botelho

I think you want something like: poetry use python3.8 to switch to 3.8, and then poetry install to create the virtual env for 3.8 – Halflength

I have no idea if the command changed over the last year, but I think the current command is poetry env use python3.8 . Source: web.archive.org/web/20210130151158/https://python-poetry.org/… – Armandoarmature

poetry env use fixed my problem. Haven't dug into details yet, but I'm guessing I initially created the poetry env (via poetry install) with 3.10, then added a .python-version for 3.11, but poetry remembered creating the venv using 3.10, and complained about the mismatch. – Mangum

Answer 4

you can specify an explicit python executable for poetry using

poetry env use <path to python executable>

This worked for me.

Broccoli

The other answers didn't work for me, but this one did. poetry is such a moving target (argh...) that I suspect the others ones are obsolete. This one deserves to be pushed to the top. – Ernestineernesto

Worked for me on Windows with multiple versions of Python installed. Just put the path in quotes and point it to whichever version you want. – Frizzy

Answer 5

On my machine I was able to fix the "currently activated Python version is not supported by the project" error by reinstalling Poetry:

curl -sSL https://install.python-poetry.org | python3 - --uninstall
curl -sSL https://install.python-poetry.org | python3 -

After that,poetry was able to find the correct version installed by pyenv.

Vondavonni

This resolved my instance of the same issue. – Tarsia

Answer 6

In my case, I had to delete and recreate the virtualenv used by poetry. This is because I added the python version restrictions (e.g. python = ">=3.6.2 <3.7") after creating the virtualenv.

Steps

• Delete the original one: run poetry env remove myApp-XkghI9p6-py3.6
• Run any poetry step, to create it, or run poetry shell, and confirm poetry run python --version is the correct version.

Thad

Answer 7

First I manually modified pyproject.toml and set

[tool.poetry.dependencies]
python = "^3.11.4"

Later in CLI

To deactivate the virtual env:

exit

To remove the virtual env

# rm -rf <path_to_virtual_env> 
# in my case:
rm -rf .venv

To set the local Python version (sets .python-version file)

pyenv local 3.11.4

To change Poetry env's Python version (TBH this might be the only command you need):

poetry env use 3.11.4

To activate a new virtual env:

poetry shell

And you should be all good to go. Check Python version with:

python -V

Unbowed

Answer 8

in my case i was running poetry via python version installed in homebrew but i tried to connect it to pyenv's python version. (since it is installed via pip i guess....)

I detected it by running:

$ where poetry

$ /opt/homebrew/bin/poetry

what worked is to remove it and install poetry again via pip in the desired version and use it the same way:

pyenv shell python3.11
python -m pip install poetry
python -m poetry install

Samp

Answer 9

Even though this issue has been resolved, I am writing this for somebody who comes across this problem again. After all attempts my python -V always resulted in 2.7 and no discussions mentioned running pyenv shell (surprising to me!) Adding pyenv to path

$ echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init -)"\nfi' >> ~/.bash_profile

In my case I had to add it to .bashrc and not bash_profile. https://ggkbase-help.berkeley.edu/how-to/install-pyenv/

Worked!

Gail

Answer 10

In my case, the environment was messed up in some way that poetry failed to activate the virtualenv properly.

Try using a different shell: perhaps, sh, or zsh. If everything works in that shell, this proves that your environment is as messed up as mine was :)

Use this command in both shells:

$ env

and try to spot the difference

Agan

Answer 11

You can remove the python version from pyproject.toml file and then run Poetry install

Clandestine

Answer 12

What worked for me was to run python3.8 -m poetry install.

Catchy

python.exe: No module named poetry – Dichasium

Answer 13

For anyone not using the venv and came looking for the answer on making poetry use specific version of python : Upgrade pip to latest version python3.x -m pip install --upgrade pip and then install poetry for that python version python3.x -m pip install poetry

Lenorelenox

Answer 14

If you installed Poetry with pipx

Things are a bit different if you installed and manage Poetry using pipx.

NOTE: Below replace the python312 command below with whatever will run the newest python, such as py or py -3.12 on windows for v3.12, similarly, replace python37 with whatever runs your older python version.

Here's my recommendation in this case:

1. Install pipx for the newest python:
   • python312 -m pip install --user pipx
   • At this point pipx will be installed to both your old and new python user folders.
2. Run pipx with list command to see what is installed. All prior installations with the old python version should still be there. The list command will tell you what Python version they were installed with. If you use the alternate way of running pipx as a module, you can force it to run with any version of python:
   • python312 -m pipx list
3. Uninstall the current poetry
   • python312 -m pipx uninstall poetry
4. Now use the newer python and pipx to install specific Python versions of poetry, using the --suffix flag to change the name of the installed file:
   • python37 -m pipx install --suffix 37 poetry
     • This will create a poetry37 entry
   • python312 -m pipx install --suffix 312 poetry
     • This will create a poetry312 entry

Although, personally, I made the poetry 3.12 version just poetry and then used poetry37 for the older python 3.7 version. Now I will run most things going forward with just poetry, but for 3.7 specific tasks I can use poetry37.

In addition, I changed the path to point to the Python 3.12 Scripts folder instead of 3.7 so the newer version of pipx could run directly at the command line without python -m pipx

Dichlorodifluoromethane

Answer 15

I had the same issue where Poetry wasn't using the Python version I set with pyenv local x.y.z. Here's the solution to ensure Poetry uses the active pyenv version:

poetry env use $(pyenv which python)

Note: this solution will work until you change the active pyenv Python version using pyenv local.

Thorbert

Answer 16

try re-installing poetry again

pip uninstall poetry
pip install poetry

Diorio