Poetry fails to install tensorflow

Question

I've got a poetry project. My environment is Conda 22.9.0 on a windows machine with poetry version 1.2.2:

This is my pyproject.toml file:

[tool.poetry]
name = "myproject"
version = "0.1.0"
description = ""

[tool.poetry.dependencies]
# REVIEW DEPENDENCIES
python = ">=3.7,<3.11"
numpy = "*"
tensorflow = "^2.8"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

[tool.poetry.scripts]
start = "myproject.main:start"

The myproject\main.py module contains:

import tensorflow as tf

def start():
    if tf.test.is_gpu_available():
        print("TensorFlow is using a GPU.")
    else:
        print("TensorFlow is NOT using a GPU.")

If I do poetry install, it seems to work fine:

Creating virtualenv myproject in D:\Projects\myproject\dev\myproject-series-forecast\.venv
Updating dependencies
Resolving dependencies...

Writing lock file

Package operations: 41 installs, 0 updates, 0 removals

• Installing certifi (2022.12.7)
• Installing charset-normalizer (2.1.1)
• Installing idna (3.4)
• Installing pyasn1 (0.4.8)
• Installing urllib3 (1.26.13)
• Installing cachetools (5.2.0)
• Installing oauthlib (3.2.2)
• Installing rsa (4.9)
• Installing six (1.16.0)
• Installing zipp (3.11.0)
• Installing requests (2.28.1)
• Installing pyasn1-modules (0.2.8)
• Installing google-auth (2.15.0)
• Installing importlib-metadata (5.2.0)
• Installing requests-oauthlib (1.3.1)
• Installing markupsafe (2.1.1)
• Installing absl-py (1.3.0)
• Installing grpcio (1.51.1)
• Installing numpy (1.21.6)
• Installing tensorboard-data-server (0.6.1)
• Installing markdown (3.4.1)
• Installing tensorboard-plugin-wit (1.8.1)
• Installing protobuf (3.19.6)
• Installing werkzeug (2.2.2)
• Installing google-auth-oauthlib (0.4.6)
• Installing astunparse (1.6.3)
• Installing flatbuffers (22.12.6)
• Installing gast (0.4.0)
• Installing google-pasta (0.2.0)
• Installing h5py (3.7.0)
• Installing keras (2.11.0)
• Installing tensorflow-estimator (2.11.0)
• Installing packaging (22.0)
• Installing opt-einsum (3.3.0)
• Installing libclang (14.0.6)
• Installing tensorboard (2.11.0)
• Installing tensorflow-io-gcs-filesystem (0.29.0)
• Installing termcolor (2.1.1)
• Installing typing-extensions (4.4.0)
• Installing wrapt (1.14.1)
• Installing tensorflow (2.11.0)

Installing the current project: myproject (0.1.0)

But when executing poetry run start I got error in import

Traceback (most recent call last):
File "<string>", line 1, in <module>
File "D:\Python\Anaconda3\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 850, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "D:\Projects\myproject\dev\myproject-series-forecast\myproject\main.py", line 3, in <module>
    import tensorflow as tf
ModuleNotFoundError: No module named 'tensorflow'

Frisse

I noticed that comparing to a different virtual environment I have in my Anaconda installation, the ./venv/Libs/site-pacakges folder under the poetry project didn't contain a /tensorflow subfolder (despite the message in poetry install stating that tensorflow was successfully installed). To solve this, I had to manually install with pip install tensorflow==2.11.0 --force-reinstall after entering the virtual env with poetry shell. Then, the module not found error disappears. – Frisse

Answer 1

For anyone dealing with the error message:

Unable to find installation candidates for tensorflow-io-gcs-filesystem (0.32.0)

It's because of the version not compatible with the tensorflow version you're trying to install (see compatibility here). For example, if you're trying to install tensorflow 2.10.*, you need to explicitly define the appropriate version of tensorflow-io-gcs-filesystem in the .toml file (i.e., tensorflow-io-gcs-filesystem = "0.27.0").

Then proceed with downloading tensorflow (i.e., poetry add tensorflow@~2.10)

Anarchist

Table of version compatibility is at pypi.org/project/tensorflow-io-gcs-filesystem – Shrive

Answer 2

I can confirm that for TF 2.11 this indeed an issue. Can't give you a definite answer as to why this no longer works, but it seems to be metadata related. Pip has no issues installing it, as already mentioned in the comments.

2.10 still works in poetry, not that that helps.

Lilla

Actually on my Mac M2 not even 2.10 works. 2.11 gives " Unable to find installation candidates for tensorflow (2.11.1)", 2.10 gives "Unable to find installation candidates for tensorflow-io-gcs-filesystem (0.32.0)" ... what a nightmare – Pustule

Answer 3

It worked for me by changing [python = ">=3.7,<3.11"] to [python = ">=3.8,<3.12"]

Danndanna

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Katharynkathe

Answer 4

For me there was a conflict when installing scikit-learn together with tensorflow. I followed the tip of @Neeraj Sharma .

before code

[tool.poetry]
name = "my-project"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.9"
jupyter = "^1.0.0"
scikit-learn = "1.2.2"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

output

➜ poetry install
Updating dependencies
Resolving dependencies... (2.5s)

The current project's Python requirement (>=3.10,<4.0) is not compatible with some of the required packages Python requirement:
  - tensorflow-io-gcs-filesystem requires Python >=3.7, <3.11, so it will not be satisfied for Python >=3.11,<4.0

Because my-project depends on tensorflow-io-gcs-filesystem (0.27.0) which requires Python >=3.7, <3.11, version solving failed.

  • Check your dependencies Python requirement: The Python requirement can be specified via the `python` or `markers` properties
    
    For tensorflow-io-gcs-filesystem, a possible solution would be to set the `python` property to ">=3.10,<3.11"

    https://python-poetry.org/docs/dependency-specification/#python-restricted-dependencies,
    https://python-poetry.org/docs/dependency-specification/#using-environment-markers

I had to look at the poetry documentation because the example is complicated to understand.

Docs Poetry Docs PEP 508

This problem we had, because it was solved with this specification of PEP 508. I didn't know it either, it ended up being a new learning experience. I recommend reading.

Solution

In [tool.poetry.dependencies] you will add the version marker, I used the one from the output. I put a slightly older version of latest. Then I added the tensorflow-io-gcs-filesystem lib specifying its version. There may be a better way to solve this, but I'm still new to poetry. It was the best solution.

[tool.poetry.dependencies]

[tool.poetry]
name = "udemy-tensorflow-developer-certificate-in-2023-zero-to-mastery"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.8,<3.11"
jupyter = "^1.0.0"
scikit-learn = "1.2.2"
tensorflow-io-gcs-filesystem = { version = "0.27.0"}
tensorflow = "2.12.0"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

After that I managed to install tensorflow together with scikit-learn.

Zicarelli

Answer 5

In my case I was using MBP M1 for which there is apparently no prebuilt binaries included by the default tensorflow package. Instead as suggested here https://github.com/python-poetry/poetry/issues/6079#issuecomment-1250119572 I had to install specific tensorflow-macos package.

Quite silly as anybody now installing the project who does not have macOS would install the wrong version. So I probably have to add some kind of hacky configuration in the pyproject.toml file.

Hardpan

Answer 6

For MBP M1 and later users, as @TeemuK mentioned, there are no tensorflow binaries provided. I had to specifically update .toml file and run poetry install. To resolve dependency issues, you can run poetry lock --no-update. This answer assumes you have poetry installed globally and already initialized.

Tree command output: Tree folder structure

pyproject.toml file:

[tool.poetry]
name = "pack"
version = "0.1.0"
description = ""
authors = ["JordanTheDodger <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = ">=3.7,<3.12"
ipykernel = {"version"="^6.29.0","python"=">=3.8,<3.12"}
pandas = {"version"="^2.1.4","python"=">=3.9,<3.12"}
matplotlib = {"version"="^3.8.2","python"=">=3.9,<3.12"}
tensorflow-macos = {"version"="2.10.0","python"=">=3.9,<3.12"}

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Disclaimer: I am using poetry virtual env in .ipnyb file. Please ignore ipykernel package. Also, tesnorflow-macos v2.10 is stable then future versions and fit for most production env

Ermines

Answer 7

Poetry Solution to tensorflow version conflict

I ran into a similar problem with my poetry-based project. Here is how I fixed it.

Its a version conflict wioth tensorflow I fixed by adding these two lines to your pyproject.toml to change your python version and the tensorflow dependency causing the issue.

[tool.poetry.dependencies]
python = ">=3.10, <3.11"
tensorflow-io-gcs-filesystem = "0.27.0"

Then run

poetry lock --no-update

and

poetry install

to see if your dependency issues are resolved.

---

This solution is based on the poetry log:

Because your_project depends on tensorflow-io-gcs-filesystem (0.27.0) which doesn't match any versions, version solving failed. poetry lock --no-update Resolving dependencies... (0.4s)

The current project's supported Python range (>=3.10,<4.0) is not compatible with some of the required packages Python requirement:

• tensorflow-io-gcs-filesystem requires Python >=3.7, <3.11, so it will not be satisfied for Python >=3.11,<4.0

Because ml depends on tensorflow-io-gcs-filesystem (0.27.0) which requires Python >=3.7, <3.11, version solving failed.

Pneumato