[2014-10 TL;DR:
distribute is dead, use pip, the new setuptools, and, for binary distributions, wheels. More below.]
[Original answer]
Distribute is was a fork of the older setuptools so nearly all comments that follow apply equally to Distribute and setuptools. Setuptools was an attempt to fill in a number of holes in the even older Python standard library package, Distutils. Setuptools added functions like automatic downloads of packages via a command-line interface, easy_install, and some level of dependency management. However, a segment of the Python community is of the opinion that setuptools is too intrusive and has too much behind-the-scenes magic for some of its features.
pip is a higher-level interface on top of setuptools or Distribute. It uses them to perform many of its functions but avoids some of their more controversial features, like zipped eggs. pip also provides features not available in setuptools, like an uninstall command and the ability to define fixed sets of requirements and reliably reproduce a set of packages. There is a more complete feature comparison here.
Why are there so many components (and there are more, like buildout)? Lots of reasons: solutions must work across all of the major platforms on which Python is supported (i.e. Unix-y, Windows, Mac OS X), so building and installation present a complex set of problems; like many open-source projects, Python is essentially all-volunteer and many developers just aren't all that interested in packaging and installation issues; there is a natural conservatism about adding major new unproven features to the standard library; differences in opinions, etc etc.
At the moment, there is a project underway to provide a replacement for Distutils and possibly for some of the higher-level add-ons. It is planned to be released in the Python 3.3 standard library as the packaging package and as an add-on for older versions of Python as Distutils2.
To summarize, the current relationship is:
pip -> [ setuptools | Distribute ] -> Distutils -> Python core
|
3rd party packages | included in Python
|
UPDATE (2012-07): Prior to feature code cutoff for Python 3.3, it was decided that packaging was not quite ready yet for release in the standard library so it has been removed from the 3.3 release. Work will continue on Distutils2 which is available via PyPI and on what will be included in the standard library for Python 3.4.
UPDATE (2014-10): There have been further changes in the world of Python packaging since this answer was last updated.
Most importantly, since mid-2013, the rift between setuptools and
distribute has been healed and development activity has been merged
into a new setuptools project. distribute is now
deprecated and no longer maintained; use the new setuptools instead
but don't use its easy_install as an installer.
pip has become the de-facto and blessed installer tool (for Python
packages not otherwise provided by your platform's package manager)
either in- or outside of virtual environments (virtualenv or pyvenv).
Instead of the old setuptools bdist eggs, wheels have
become the blessed binary distribution format for Python packages.
As of Python 3.4, a version of pip with wheel support
is now shipped with the official python.org binary installers and
source packages and it is anticipated that pip will also be
included in the next maintenance release of Python 2.7 (2.7.9).
Distutils2 and packaging are now dormant.
More details in the new Distributing Python Modules section of the Python 3 docs and the new Python Packaging User Guide.
