Pip freeze vs. pip list

Question

Why does pip list generate a more comprehensive list than pip freeze?

$ pip list
feedparser (5.1.3)
pip (1.4.1)
setuptools (1.1.5)
wsgiref (0.1.2)

$ pip freeze
feedparser==5.1.3
wsgiref==0.1.2

---

Pip's documentation states:

freeze	Output installed packages in requirements format.
list	List installed packages.

What is a "requirements format"?

Patchy

FYI, here are the docs. – Confectionery

Up to date documentation as of now. – Overthecounter

Answer 1

One may generate a requirements.txt via:

$ pip freeze > requirements.txt

A user can use this requirements.txt file to install all the dependencies. For instance:

$ pip install -r requirements.txt

The packages need to be in a specific format for pip to understand, such as:

# requirements.txt
feedparser==5.1.3
wsgiref==0.1.2
django==1.4.2
...

That is the "requirements format".

Here, django==1.4.2 implies install django version 1.4.2 (even though the latest is 1.6.x). If you do not specify ==1.4.2, the latest version available would be installed.

You can read more in "Virtualenv and pip Basics", and the official "Requirements File Format" documentation.

Assessor

Got it. Is there any particular reason why "list" produces a more comprehensive list than "freeze"? – Patchy

i think it is because pip list lists everything, and pip freeze installs everything installed by pip. – Assessor

Hm, that's a theory, but I'm fairly sure I didn't pip install wsgiref. – Patchy

Python 3.2 includes wsgiref.egg-info in the Lib directory, which is why pip knows about it. You can't [un]install it with pip, and later versions of Python omit the metadata file so it won't appear. – Liebig

you can use requirements.txt regardless of whether you're using a virtualenv or not – Loaiasis

@Patchy some of the packages you see there were installed as dependencies to others – Loaiasis

Is there a way to install the latest versions of the packages and not the specific ones, and is it a good idea to do so? – Tideland

@leonid if you get rid of the version number at the end , it would install the latest version from pypi. Read the answer completely – Assessor

This answer doesn't address the most puzzling part of the question - why do the two lists differ? It only addresses the self-evident part of the question - requirements format is the format output by 'pip freeze', an example of which is shown in the question. – Fuel

Answer 2

To answer the second part of this question, the two packages shown in pip list but not pip freeze are setuptools (which is easy_install) and pip itself.

It looks like pip freeze just doesn't list packages that pip itself depends on. You may use the --all flag to show also those packages.

From the documentation:

--all

Do not skip these packages in the output: pip, setuptools, distribute, wheel

Burnett

pip freeze has an option --all to show also: pip, setuptools, distribute, wheel pip.pypa.io/en/stable/reference/pip_freeze – Demello

less distribute – Ahmednagar

Answer 3

The main difference is that the output of pip freeze can be dumped into a requirements.txt file and used later to re-construct the "frozen" environment.

In other words you can run: pip freeze > frozen-requirements.txt on one machine and then later on a different machine or on a clean environment you can do: pip install -r frozen-requirements.txt and you'll get the an identical environment with the exact same dependencies installed as you had in the original environment where you generated the frozen-requirements.txt.

Hospitalization

Answer 4

Look at the pip documentation, which describes the functionality of both as:

pip list

List installed packages, including editables.

pip freeze

Output installed packages in requirements format.

So there are two differences:

1. Output format, freeze gives us the standard requirement format that may be used later with pip install -r to install requirements from.

2. Output content, pip list include editables which pip freeze does not.

Microelectronics

pip.pypa.io/en/stable/reference/pip_install/#install-editable – Microelectronics

Answer 5

My preferred method of generating a requirements file is:

pip list --format=freeze > requirements.txt

This method keeps just the package names and package versions without potentially linking to local file paths which 'pip freeze' alone will sometimes give me. Local file paths in a requirements file make your codebase harder to use for other users and some developers don't know how to fix this so I prefer this method for ease of adoptability.

Gerous

Wouldn't pip list --not-required --format=freeze > requirements.txt be even better since it only includes the top-level dependencies? – Oliver

@Oliver thanks for your input! I haven't done a deep dive into using this flag but I think it probably makes a requirements file cleaner and easier to understand. I am seeing some issues reported for it missing some particular dependencies which could be a potential issue but probably few and far between. Overall I'm in favor of explicit environment configurations and minimizing nondeterministic decisions so I would say in a data quality critical application like scientific computing it's useful to list all dependencies, but for most applications, top level is good enough. – Gerous

Yeah, it may exclude some top-level dependencies if they are also nested in another one. The only problem I see with explicit dependencies all the way down is it's difficult to upgrade and you may have extra ones that are no longer needed when nested dependencies change. – Oliver

I personally find this the most useful command for the reasons noted. Explicit and flexible between package managers – Conative

Answer 6

pip list shows ALL installed packages.

pip freeze shows packages YOU installed via pip (or pipenv if using that tool) command in a requirements format.

Remark below that setuptools, pip, wheel are installed when pipenv shell creates my virtual envelope. These packages were NOT installed by me using pip:

test1 % pipenv shell
Creating a virtualenv for this project…
Pipfile: /Users/terrence/Development/Python/Projects/test1/Pipfile
Using /usr/local/Cellar/pipenv/2018.11.26_3/libexec/bin/python3.8 (3.8.1) to create virtualenv…
⠹ Creating virtual environment...
<SNIP>
Installing setuptools, pip, wheel...
done.
✔ Successfully created virtual environment! 
<SNIP>

Now review & compare the output of the respective commands where I've only installed cool-lib and sampleproject (of which peppercorn is a dependency):

test1 % pip freeze       <== Packages I'VE installed w/ pip

-e git+https://github.com/gdamjan/hello-world-python-package.git@10<snip>71#egg=cool_lib
peppercorn==0.6
sampleproject==1.3.1


test1 % pip list         <== All packages, incl. ones I've NOT installed w/ pip

Package       Version Location                                                                    
------------- ------- --------------------------------------------------------------------------
cool-lib      0.1  /Users/terrence/.local/share/virtualenvs/test1-y2Zgz1D2/src/cool-lib           <== Installed w/ `pip` command
peppercorn    0.6       <== Dependency of "sampleproject"
pip           20.0.2  
sampleproject 1.3.1     <== Installed w/ `pip` command
setuptools    45.1.0  
wheel         0.34.2

Laurencelaurene

pip list - shows installed packages, not ALL. Check official documentation and the answer from Daniel Lahyani. – Oarsman

@filler36: I thought that "installed" was implied, but nonetheless incorporated your feedback to ensure the ambiguity didn't create a potential confusion. Thanks for your feedback, very much obliged!- T – Laurencelaurene

"pip freeze shows packages YOU installed via pip". On Linux this is WRONG! Both list and freeze show ALL packages, including those installed by the system's package manager (which is NOT YOU). pip freeze omits only (setuptools, wheel, pip, distribute), but has an all option to include them. documentation. – Firewater

Answer 7

pip list

List installed packages: show ALL installed packages that even pip installed implictly

pip freeze

List installed packages: - list of packages that are installed using pip command

pip freeze has --all flag to show all the packages.

Other difference is the output it renders, that you can check by running the commands.

Livia

On Linux, both list and freeze show ALL packages, including those installed by the system's package manager (i.e. not installed by pip or by you). pip freeze omits ONLY (setuptools, wheel, pip, distribute). – Firewater

Answer 8

For those looking for a solution. If you accidentally made pip requirements with pip list instead of pip freeze, and want to convert into pip freeze format. I wrote this R script to do so.

library(tidyverse)

pip_list = read_lines("requirements.txt")

pip_freeze = pip_list %>%
  str_replace_all(" \\(", "==") %>%
  str_replace_all("\\)$", "")

pip_freeze %>% write_lines("requirements.txt")

Metrist