PyLance in Visual Studio Code does not recognise Poetry virtual env dependencies

Question

I am using Poetry to manage a Python project. I create a virtual environment for Poetry using a normal poetry install and pyproject.toml workflow. Visual Studio Code and its PyLance does not pick up project dependencies in Jupyter Notebook.

• Python stdlib modules are recognised
• The modules of my application are recognised
• The modules in the dependencies and libraries my application uses are not recognised

Instead, you get an error

Import "xxx" could not be resolved Pylance (reportMissingImports)

An example screenshot with some random imports that show what is recognised and what is not (tradeexecutor package is Poetry project, then some random Python packages dependency are not recognised).:

The notebook still runs fine within Visual Studio Code, so the problem is specific to PyLance, the virtual environment is definitely correctly set up.

Some Python Language Server output (if relevant):

2024-03-01 10:15:40.628 [info] [Info  - 10:15:40] (28928) Starting service instance "trade-executor"
2024-03-01 10:15:40.656 [info] [Info  - 10:15:40] (28928) Setting pythonPath for service "trade-executor": "/Users/moo/code/ts/trade-executor"
2024-03-01 10:15:40.657 [info] [Info  - 10:15:40] (28928) Setting environmentName for service "trade-executor": "3.10.13 (trade-executor-8Oz1GdY1-py3.10 venv)"
2024-03-01 10:15:40.657 [info] [Info  - 10:15:40] (28928) Loading pyproject.toml file at /Users/moo/code/ts/trade-executor/pyproject.toml
2024-03-01 10:15:40.657 [info] [Info  - 10:15:40] (28928) Pyproject file "/Users/moo/code/ts/trade-executor/pyproject.toml" has no "[tool.pyright]" section.
2024-03-01 10:15:41.064 [info] [Info  - 10:15:41] (28928) Found 763 source files
2024-03-01 10:15:41.158 [info] [Info  - 10:15:41] (28928) Background analysis(4) root directory: file:///Users/moo/.vscode/extensions/ms-python.vscode-pylance-2024.2.2/dist
2024-03-01 10:15:41.158 [info] [Info  - 10:15:41] (28928) Background analysis(4) started
2024-03-01 10:15:41.411 [info] [Info  - 10:15:41] (28928) Indexer background runner(5) root directory: file:///Users/moo/.vscode/extensions/ms-python.vscode-pylance-2024.2.2/dist (index)
2024-03-01 10:15:41.411 [info] [Info  - 10:15:41] (28928) Indexing(5) started
2024-03-01 10:15:41.662 [info] [Info  - 10:15:41] (28928) scanned(5) 1 files over 1 exec env
2024-03-01 10:15:42.326 [info] [Info  - 10:15:42] (28928) indexed(5) 1 files over 1 exec

Also looks like PyLance correctly finds the virtual environment in the earlier Python Language Server output:

2024-03-03 19:36:56.784 [info] [Info  - 19:36:56] (41658) Pylance language server 2024.2.2 (pyright version 1.1.348, commit cfb1de0c) starting
2024-03-03 19:36:56.789 [info] [Info  - 19:36:56] (41658) Server root directory: file:///Users/moo/.vscode/extensions/ms-python.vscode-pylance-2024.2.2/dist
2024-03-03 19:36:56.789 [info] [Info  - 19:36:56] (41658) Starting service instance "trade-executor"
2024-03-03 19:36:57.091 [info] [Info  - 19:36:57] (41658) Setting pythonPath for service "trade-executor": "/Users/moo/Library/Caches/pypoetry/virtualenvs/trade-executor-8Oz1GdY1-py3.10/bin/python"
2024-03-03 19:36:57.093 [info] [Info  - 19:36:57] (41658) Setting environmentName for service "trade-executor": "3.10.13 (trade-executor-8Oz1GdY1-py3.10 venv)"
2024-03-03 19:36:57.096 [info] [Info  - 19:36:57] (41658) Loading pyproject.toml file at /Users/moo/code/ts/trade-executor/pyproject.toml

How to diagnose the issue further and then fix the issue?

Latreshia

Are you using remote connection? Probably related: in github: github.com/microsoft/vscode-jupyter/issues/15088 – Kaminski

Not using a remote connection, but good to know! – Latreshia

PyLance is based on Pyright. You might need to create a pyrightconfig.json file at the root of your project with the following content to recognize the Poetry virtual environment. – Nicias

@Nicias what following content? – Evangelia

@Evangelia With venvpath et venv, in case the issue is the Poetry project needs to find imports in the virtual environment’s site-packages directory rather than the paths specified by the default Python interpreter. – Nicias

Thank you @Nicias ! Do you have any links to documentation or examples regarding how to accomplish this? – Latreshia

@MikkoOhtamaa Just a pyrightconfig.json file at the root of your project with the JSON { "venvPath": "<path to the folder containing your .venv>", "venv": ".venv"} – Nicias

Gotcha. I will try this and report back results today. – Latreshia

I was expecting if Visual Studio Code configures PyLance internally it would do this for me, and I do not need to mess with Visual Studio Code's internal process. – Latreshia

This could be related github.com/microsoft/vscode-python/issues/… – Appertain

@Appertain but this question isn't about using the terminal. it's about a jupyter notebook. – Evangelia

Answer 1

Some other people were having the same issue.

• Pylance does not show auto import information from site-packages directory #3281
• Pylance not indexing all files and symbols for sqlalchemy even with package depth of 4

Visual Studio Code's PyLance implementation seems to have some internal limits that may prevent indexing all files. However, this was not the case for me. Instead, PyLance was somehow corrupted.

Running: PyLance: Clear all persistent indices from the command palette fixed the issue for. After this, PyLance seemed to behave.

Latreshia

Answer 2

Here is another answer on when Jupyter Notebook, Visual Studio Code and Poetry fail:

If you use pyproject.toml with Poetry's develop = true feature, It just does not work, as Visual Studio Code and Pylance does not support it.

• I tried manually force Pylance to scan the packages through various settings, but Pylance still refused to scan the packages

Here is the pyproject.toml workaround:

#
# TODO: It's very hard to make local dev env to have trade-executor
# editable dependency with Poetry at the moment.
# Thus we assume is that if you install yourself, you have cloned
# trade-executor to ../trade-executor path.
# Or change the line below.
#
# Poetry develop = true and Visual Studio Code bug https://github.com/microsoft/pylance-release/issues/4664
#
[tool.poetry.dependencies]
python = ">=3.10,<3.12"
trade-executor = {path = "../trade-executor", develop = false, extras=["execution", "quantstats"]}
ipython = "^8.12.0"
ipdb = "^0.13.13"
parquet-cli = "^1.3"

Latreshia