Does Python have a package/module management system?
Asked Answered
B

13

146

Does Python have a package/module management system, similar to how Ruby has rubygems where you can do gem install packagename?

On Installing Python Modules, I only see references to python setup.py install, but that requires you to find the package first.

Beera answered 13/3, 2010 at 0:29 Comment(2)
If you think Python's package index PyPI should have clearer instructions how to install packages, please vote and comment on the issue at bitbucket.org/pypa/pypi/issue/149/…Raychel
While it was current at the time, the link in this question is now to the legacy distutils documentation. I filed bugs.python.org/issue26014 to cover that these docs need to be clearer that they're no longer suitable for use as end user documentation (they're solely kept around for the benefit of tool implementors).Savior
R
214

Recent progress

March 2014: Good news! Python 3.4 ships with Pip. Pip has long been Python's de-facto standard package manager. You can install a package like this:

pip install httpie

Wahey! This is the best feature of any Python release. It makes the community's wealth of libraries accessible to everyone. Newbies are no longer excluded from using community libraries by the prohibitive difficulty of setup.

However, there remains a number of outstanding frustrations with the Python packaging experience. Cumulatively, they make Python very unwelcoming for newbies. Also, the long history of neglect (ie. not shipping with a package manager for 14 years from Python 2.0 to Python 3.3) did damage to the community. I describe both below.

Outstanding frustrations

It's important to understand that while experienced users are able to work around these frustrations, they are significant barriers to people new to Python. In fact, the difficulty and general user-unfriendliness is likely to deter many of them.

PyPI website is counter-helpful

Every language with a package manager has an official (or quasi-official) repository for the community to download and publish packages. Python has the Python Package Index, PyPI. https://pypi.python.org/pypi

Let's compare its pages with those of RubyGems and Npm (the Node package manager).

  1. https://rubygems.org/gems/rails RubyGems page for the package rails
  2. https://www.npmjs.org/package/express Npm page for the package express
  3. https://pypi.python.org/pypi/simplejson/ PyPI page for the package simplejson

You'll see the RubyGems and Npm pages both begin with a one-line description of the package, then large friendly instructions how to install it.

Meanwhile, woe to any hapless Python user who naively browses to PyPI. On https://pypi.python.org/pypi/simplejson/ , they'll find no such helpful instructions. There is however, a large green 'Download' link. It's not unreasonable to follow it. Aha, they click! Their browser downloads a .tar.gz file. Many Windows users can't even open it, but if they persevere they may eventually extract it, then run setup.py and eventually with the help of Google setup.py install. Some will give up and reinvent the wheel..

Of course, all of this is wrong. The easiest way to install a package is with a Pip command. But PyPI didn't even mention Pip. Instead, it led them down an archaic and tedious path.

Error: Unable to find vcvarsall.bat

Numpy is one of Python's most popular libraries. Try to install it with Pip, you get this cryptic error message:

Error: Unable to find vcvarsall.bat

Trying to fix that is one of the most popular questions on Stack Overflow: "error: Unable to find vcvarsall.bat"

Few people succeed.

For comparison, in the same situation, Ruby prints this message, which explains what's going on and how to fix it:

Please update your PATH to include build tools or download the DevKit from http://rubyinstaller.org/downloads and follow the instructions at http://github.com/oneclick/rubyinstaller/wiki/Development-Kit

Publishing packages is hard

Ruby and Nodejs ship with full-featured package managers, Gem (since 2007) and Npm (since 2011), and have nurtured sharing communities centred around GitHub. Npm makes publishing packages as easy as installing them, it already has 64k packages. RubyGems lists 72k packages. The venerable Python package index lists only 41k.

History

Flying in the face of its "batteries included" motto, Python shipped without a package manager until 2014.

Until Pip, the de facto standard was a command easy_install. It was woefully inadequate. The was no command to uninstall packages.

Pip was a massive improvement. It had most the features of Ruby's Gem. Unfortunately, Pip was--until recently--ironically difficult to install. In fact, the problem remains a top Python question on Stack Overflow: "How do I install pip on Windows?"

Raychel answered 18/11, 2012 at 23:27 Comment(10)
For anybody interested, there PEP 439 is proposing a way to streamline the pip install process, although I think this PEP is just part of a wider strategy that's being looked at. See also discussion here.Impoverish
@Impoverish that's great news! Python deserves a package manager, I hope the devs can make it happen.Raychel
Absolutely, long overdue. I definitely hope that it makes it into Python 3.4, at least. Unfortunately I very much doubt that it'll end up being back-ported to 2.x, but there is at least light at the end of the tunnel. Let's just hope it's not an oncoming dragon... (^_^)Impoverish
In Python's defense, newer languages such as Ruby and Node were fortunate to be able to observe the mistakes of previous languages and avoid them. That said, I really wish package management on Python would receive more love and care, so I'm glad to see PEP 439 as pointed out by @Cartroo.Castrate
I agree that Ruby's package management is a wonderful thing from my (end user) perspective.. gem install X just seems to work on all platforms. So how did they manage this for gems which require C code compilation on Windows?Grown
@Grown if you try to install a Gem with C extensions but don't have a compiler, you'll get a message asking you to install Ruby DevKit, which is made as simple possible. Better than Python's unintelligible 'unable to find vcvarsall.bat'Raychel
pip can be easily installed on Windows (incl. 64bit) using the instructions hereKial
For those following along at home. On Arch Linux, pip being bundled with Python 3.4+ still means you have to install python-pip through pacman.Kathrynkathryne
Packagist and composer probably deserve a place in the comparison list now.Kerk
This doens't seem true if you're using a Mac which comes with Python 2.7 pre-installed. Even after installing Python 3 (using GUI) running the pip command installs for Python 2.7.Limp
R
63

And just to provide a contrast, there's also pip.

Redfield answered 13/3, 2010 at 0:33 Comment(3)
thanks! why isn't pip mentioned on the installing python modules documentation page?Beera
Note that pip doesn't support eggs, it only installs source packages. Generally speaking, pip does many things much better than easy_install, but easy_install is likely to be installed by default on most unixes, pip is not.Kymric
@Tommy Right! If the Python devs cared about user experience, they'd ship Python with a package manager (such as pip). Ruby ships with Gem, and Nodejs with Npm. https://mcmap.net/q/12907/-does-python-have-a-package-module-management-systemRaychel
K
63

The Python Package Index (PyPI) seems to be standard:

  • To install a package: pip install MyProject
  • To update a package pip install --upgrade MyProject
  • To fix a version of a package pip install MyProject==1.0

You can install the package manager as follows:

curl -O http://python-distribute.org/distribute_setup.py
python distribute_setup.py
easy_install pip

References:

Keyhole answered 2/9, 2012 at 9:13 Comment(2)
This is the modern answer.Mordy
I searched the internet for 20 minutes and this answer was nowhere. Thank you so much. Also, I have serious doubts about using python if the documentation is this bad...this should be on the Python main page or accesible from google or somethingOrientate
C
21

As a Ruby and Perl developer and learning-Python guy, I haven't found easy_install or pip to be the equivalent to RubyGems or CPAN.

I tend to keep my development systems running the latest versions of modules as the developers update them, and freeze my production systems at set versions. Both RubyGems and CPAN make it easy to find modules by listing what's available, then install and later update them individually or in bulk if desired.

easy_install and pip make it easy to install a module ONCE I located it via a browser search or learned about it by some other means, but they won't tell me what is available. I can explicitly name the module to be updated, but the apps won't tell me what has been updated nor will they update everything in bulk if I want.

So, the basic functionality is there in pip and easy_install but there are features missing that I'd like to see that would make them friendlier and easier to use and on par with CPAN and RubyGems.

Corgi answered 13/3, 2010 at 1:31 Comment(1)
This answer is now well and truly outdated.Savior
B
13

There are at least two, easy_install and its successor pip.

Biquadrate answered 13/3, 2010 at 0:34 Comment(0)
B
11

As of at least late 2014, Continuum Analytics' Anaconda Python distribution with the conda package manager should be considered. It solves most of the serious issues people run into with Python in general (managing different Python versions, updating Python versions, package management, virtual environments, Windows/Mac compatibility) in one cohesive download.

It enables you to do pretty much everything you could want to with Python without having to change the system at all. My next preferred solution is pip + virtualenv, but you either have to install virtualenv into your system Python (and your system Python may not be the version you want), or build from source. Anaconda makes this whole process the click of a button, as well as adding a bunch of other features.

Berlinda answered 17/12, 2014 at 23:47 Comment(0)
L
8

That'd be easy_install.

Labyrinth answered 13/3, 2010 at 0:31 Comment(1)
If you want to downvote everyone on the page who said easy_install, please check the dates first. That was valid advice when it was given almost 7 years ago. I don't think of of us answerers are going to regularly revisit this to update, so downvoting is silly.Labyrinth
T
6

It's called setuptools. You run it with the "easy_install" command.

You can find the directory at http://pypi.python.org/

Treillage answered 13/3, 2010 at 0:32 Comment(0)
W
5

I don't see either MacPorts or Homebrew mentioned in other answers here, but since I do see them mentioned elsewhere on Stack Overflow for related questions, I'll add my own US$0.02 that many folks seem to consider MacPorts as not only a package manager for packages in general (as of today they list 16311 packages/ports, 2931 matching "python", albeit only for Macs), but also as a decent (maybe better) package manager for Python packages/modules:

Question

"...what is the method that Mac python developers use to manage their modules?"

Answers

SciPy

"Macs (unlike Linux) don’t come with a package manager, but there are a couple of popular package managers you can install. Macports..."

I'm still debating on whether or not to use MacPorts myself, but at the moment I'm leaning in that direction.

Whistling answered 12/1, 2013 at 2:40 Comment(1)
Homebrew itself says: You can install Python packages with (the outdated easy_install or) pip install <your_favorite_package>Odum
J
4

On Windows install http://chocolatey.org/ then

choco install python

Open a new cmd-window with the updated PATH. Next, do

choco install pip

After that you can

pip install pyside
pip install ipython
...
Jim answered 6/6, 2014 at 16:11 Comment(1)
I wanted to answer this as a comment to Thomas Bratt's answer but I don't have enough reputation...Jim
E
3

Since no one has mentioned pipenv here, I would like to describe my views why everyone should use it for managing python packages.

As @ColonelPanic mentioned there are several issues with the Python Package Index and with pip and virtualenv also.

Pipenv solves most of the issues with pip and provides additional features also.

Pipenv features

Pipenv is intended to replace pip and virtualenv, which means pipenv will automatically create a separate virtual environment for every project thus avoiding conflicts between different python versions/package versions for different projects.

  • Enables truly deterministic builds, while easily specifying only what you want.
  • Generates and checks file hashes for locked dependencies.
  • Automatically install required Pythons, if pyenv is available.
  • Automatically finds your project home, recursively, by looking for a Pipfile.
  • Automatically generates a Pipfile, if one doesn’t exist.
  • Automatically creates a virtualenv in a standard location.
  • Automatically adds/removes packages to a Pipfile when they are un/installed.
  • Automatically loads .env files, if they exist.

If you have worked on python projects before, you would realize these features make managing packages way easier.

Other Commands

  • check checks for security vulnerabilities and asserts that PEP 508 requirements are being met by the current environment. (which I think is a great feature especially after this - Malicious packages on PyPi)
  • graph will show you a dependency graph, of your installed dependencies.

You can read more about it here - Pipenv.

Installation

You can find the installation documentation here

P.S.: If you liked working with the Python Package requests , you would be pleased to know that pipenv is by the same developer Kenneth Reitz

Emperor answered 14/12, 2017 at 17:4 Comment(0)
U
0

In 2019 poetry is the package and dependency manager you are looking for.

https://github.com/sdispater/poetry#why

It's modern, simple and reliable.

Uziel answered 4/1, 2019 at 14:30 Comment(0)
V
0

Poetry is what you're looking for. It takes care of dependency management, virtual environments, running.

Valli answered 22/2, 2019 at 20:29 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.