Download dependencies declared in pyproject.toml using Pip

Question

I have a Python project that doesn't contain requirements.txt. But it has a pyproject.toml file.

How can I download packages (dependencies) required by this Python project and declared in pyproject.toml using the Pip package manager (instead of the build tool Poetry).

So instead of pip download -r requirements.txt, something like pip download -r pyproject.toml.

Tumbrel

#35064926 – Wojcik

Answer 1

Here is an example of .toml file:

[build-system]
requires = [
    "flit_core >=3.2,<4",
]
build-backend = "flit_core.buildapi"

[project]
name = "aedttest"
authors = [
    {name = "Maksim Beliaev", email = "[email protected]"},
    {name = "Bo Yang", email = "[email protected]"},
]
readme = "README.md"
requires-python = ">=3.7"
classifiers = ["License :: OSI Approved :: MIT License"]
dynamic = ["version", "description"]

dependencies = [
    "pyaedt==0.4.7",
    "Django==3.2.8",
]

[project.optional-dependencies]
test = [
    "black==21.9b0",
    "pre-commit==2.15.0",
    "mypy==0.910",
    "pytest==6.2.5",
    "pytest-cov==3.0.0",
]

deploy = [
    "flit==3.4.0",
]

to install core dependencies you run:

pip install .

if you need test(develop) environment (we use test because it is a name defined in .toml file, you can use any):

pip install .[test]

To install from Wheel:

pip install C:\git\aedt-testing\dist\aedttest-0.0.1-py3-none-any.whl[test]

Candracandy

pip install . doesn't just download the dependencies, it installs them, as well as the actual project itself. – Erethism

true, but maybe this answer will be anyway helpful for somebody. I find limited info on pyproject.toml file – Candracandy

This answer was certainly helpful for me - I couldn't find anywhere else the 'dependencies' list under project was specified, and that's what is needed for pip to install other packages needed at run time. – Moffett

This is a great example, and like @A. Rose I couldn't find much in the way of examples. However I have one question/problem remaining (and I accept that I may be searching for a tree while stumbling through a forest): how do I install the project.optional-dependencies from your example using pip? – Ascidium

Ha! Answered my own question (I found that dang tree!). From here I deduced that installing an [project.optional-dependencies] is done like so: python -m pip install .[dependency-name]. – Ascidium

If you run Zsh, run pip install -e '.[tests]', if you get the zsh: no matches found: .[tests] error. – Duckett

What about listing a local package in the toml dependencies. There's a package I need that is not on pypi and I have included its repo in my project. I've tried "/packagedirectory" and other similar syntax but none of these are PEP 508 compliant and thus raise an error. Hints? – Book

Never mind, I came up with a solution to local dependencies with "relative" absolute paths here. – Book

@Duckett On zsh you can run pip install -e .'[tests]' with the dot outside the quotes and that should work, at least it works for me with zsh in iTerm2 on my Mac. – Wojcik

Answer 2

pip supports installing pyproject.toml dependencies natively.

As of version 10.0, pip supports projects declaring dependencies that are required at install time using a pyproject.toml file, in the form described in PEP 518. When building a project, pip will install the required dependencies locally, and make them available to the build process. Furthermore, from version 19.0 onwards, pip supports projects specifying the build backend they use in pyproject.toml, in the form described in PEP 517.

From the project's root, use pip's local project install:

python -m pip install .

Hydrophobia

That command doesn't just download the dependencies, it installs them, as well as the actual project itself. – Erethism

is there a way to point to some pyproject.toml file outside of a local project? – Sipe

Answer 3

If you want a lighting fast version of pip-tools try uv

uv pip compile pyproject.toml --all-extras > requirements-dev.txt
uv pip install -r requirements-dev.txt

Gluconeogenesis

Answer 4

For whoever is looking for a simple answer with only pip, this worked for me:

pip install . && pip uninstall -y <your-project>

E.g. I have a Dockerfile looks like this:

ADD pyproject.toml /workspaces/my-awesome-project/

RUN pip install . && \
    pip uninstall -y my-awesome-project && \
    rm pyproject.toml

Respect

I don't know why this was downvoted. This does temporarily install the package too, but this is one of the few answers actually answering the question as asked. – Radburn

Answer 5

If you want to download the dependencies you can do the following:

1. cd into the directory where the pyproject.toml file is contained
2. pip download .

Quadrature

Answer 6

You can export the dependencies to a requirements.txt and use pip download afterwards:

poetry export -f requirements.txt > requirements.txt
pip download -r  requirements.txt

Autoerotism

The question is about using the pyproject.toml file directly, not generating a requirements.txt file. – Moffett

The OP said they are using Poetry. So this is the answer to do it. If you really want to have only one command, the answer would be: You cannot. – Autoerotism

The OP asks specifically about using pip to download the dependencies directly using the pyproject.toml file - which pip can do (as it can interpret the given pyproject.toml file). The poetry export is not needed. – Moffett

The OP said they are not using poetry. But I agree it is confusing, because python-poetry is tagged. – Tuff

Upvoting this. This is not a perfect solution, far from it, but it does allow for easy extraction of requirements without installing the project itself – Fabaceous

Answer 7

The comment by @andrew (When would the -e, --editable option be useful with pip install?) is a clue but I'll make it explicit. Simply doing

pip install -e .

will install the dependencies. It also "installs" the project as an "editable install". That installs only metadata about the current project - it doesn't generate build output or a whl, and changes in the source files are immediately reflected in the "installed" version.

Jehiah