PIP: Installing only the dependencies

Question

I have a script that creates a virtualenv, installs distribute and pip in it and then optionally clones a git repo.

Now I have the project I will be working on, installed. But its dependencies are not installed. How can I make pip install all the dependencies as if I have issued a pip install MyApp?

EDIT: Appareantly my question is a duplicate of this one.

Not exactly sure but pip install -e . seems to do what I want without too many extra stuff lying around. I'd prefer if my code wasn't linked from site-packages though.

Peacemaker

I have a script that creates a virtualenv, installs distribute and pip in it (...). You know you can do all 3 things with pip -E VENV_DIR pip as pip by default installs pip and distribute in the newly created environment? – Reconstructive

Cool tip! I don't use the mentioned script anymore though. – Peacemaker

perhaps useful if you want to install only depedencies with pip using setup.py: #30797624 – Telekinesis

see my answer here: https://mcmap.net/q/296742/-how-to-use-setup-py-to-install-dependencies-only – Telekinesis

do python setup.py egg_info; pip install -r *.egg-info/requires.txt; rm -rf *.egg-info/ – Telekinesis

Answer 1

In my package root issuing pip install -e . installs dependencies.

Peacemaker

That's some pretty serious side-effecting. -e installs in editable mode, which means the package gets linked from site packages instead of copied. I'd expect this approach to cause weird and subtle problems when you go and try to install the package for real, especially if you don't pip uninstall it first. – Repro

Note that this tries to install the package itself too in some way. See github.com/pypa/pip/issues/7218 for an example where this has unwanted effects. – Arevalo

this is obviously not the right answer! Why is this accepted? This will do much more than "only install dependencies". – Telekinesis

@maciek since pip v21.3 editable installs when using [only] a pyproject.toml see: pip.pypa.io/en/stable/reference/build-system/pyproject-toml/… – Semimonthly

@CharlieParker while it doesn't answer the question, it solves the OP's problem – Semimonthly

Answer 2

If your dependencies are defined in the setup.py file, you can first dump them to an external file using:

python setup.py egg_info

This will list all your dependencies in YOUR_PROJECT.egg-info/requires.txt file. Then you can install them using pip:

pip install -r *.egg-info/requires.txt

to delete what you just created:

rm -rf *.egg-info/

to save some time copy pasting:

python setup.py egg_info
pip install -r *.egg-info/requires.txt
rm -rf *.egg-info/

Glenoid

this will not work if you have extra_require because the requires.txt will contain [dev] – Klutz

If your setup.py has extras_require you may be able to sed out the extra lines before calling pip install -r – Triode

is there a way to skip dumping them into an external file? perhaps you should tell us too how to remove all the extra files etc that the first command you have does python setup.py egg_info. – Telekinesis

+1 on the wild card to make the pip install work with everyone. Only missing the rm -rf *.egg-info/ which I'd like to personally run to remove the dep file. fyi my answer inspired from this https://mcmap.net/q/296742/-how-to-use-setup-py-to-install-dependencies-only – Telekinesis

what if my dependencies are defined in pyproject.toml? – Semimonthly

@Semimonthly then create a wrapper setup.py with just from setuptools import setup setup() and run the command. Make sure your setuptools and pip are updated – Brazil

@Brazil @Semimonthly If your project only contains a pyproject.toml, there is no need to create a temporary setup.py file. You can just call Python code directly, e.g. like python -c 'from setuptools import setup; setup(script_args=["egg_info"])' -- this is inevitably the same as what @Brazil suggested above. – Sylviesylvite

Answer 3

To install your project's dependencies (i.e. install_requires + extra_requires) you have to extract your dependencies using setuptools egg-info and then install the filtered list of the combined dependencies:

python setup.py egg_info
pip install `grep -v '^\[' *.egg-info/requires.txt`

Melanite

Answer 4

You can use pip-tools to create a requirements.txt that only contains the dependencies of your package:

$ pip-compile -o requirements.txt setup.py

Note that the command above only works if you do not already have a requirements.txt file. If you happen to have one already, just delete it.

Using the generated requirements.txt you can then run pip to install the dependencies:

$ pip install -r requirements.txt

Bonus 1:

The requirements.txt will include comments that indicate where the regarding dependency originates from.

Bonus 2:

If you have have an extras_require section for optional dependencies in your setup.py that looks e.g. like this:

    ...
    extras_require={
        "development": [
            "wheel",
            "debugpy",
            "pytest",
        ],
    },
    ...

You can create the requirements.txt including the optional dependencies by using:

$ pip-compile -o requirements.txt --extra development setup.py

Nyala

Answer 5

You should use the pip requirements file.

Essentially, place all your requirements, one in each line in a file and pass that to pip using the command

pip install -r requirements.txt

What more, if you have a standard environment, pip can actually dump such a file from existing installs using the command:

pip freeze

You can put the file thus generated directly into the pip requirements, and call the previous command from your deployment script.

Pretty cool, isnt it? :)

Tortoni

Cool, but it doesn't answer my question. I'm not looking for a way to define dependencies. Reading questions entirely and carefully helps goes a long way in providing correct answers. – Peacemaker

Wait, You can put all your dependencies in a file and ask pip to install them all for you. Isn't that what you are looking for? If not, I didn't properly understand your question. Even now. – Tortoni

the difference being where the 'dependencies' are specified - in the requirements.txt (where you would specify "myrepo==1.0.1" or whatever) OR in the setup.py of the package in myrepo that you want to install. if you've already got the repo in hand then only installing the dependencies might make sense. – Plat

@Tgr I don't think that is correct. I just used pip3 to install via a requirements file and several transitive dependencies were installed. – Mask

@Mask this definitely used to be a problem some time ago. I have run into it with a recent version of pip as well, but it's entirely possible that that was some kind of user error on my side. – Baloney

The question is asking for a way to install the dependencies of the package being developed. Its dependencies will be declared in the package's setup.py file. It could even have conditional dependencies (like based on OS). I don't think requirements.txt can handle that, but even if it can, you'd have to do some extra work to load the requirements into setup.py from the file. – Packsaddle

This is not quite right, because setup.py and requirements.txt are for different purposes and their roles are not interchangeable. See Python packaging document: packaging.python.org/discussions/… – Roots