What's the proper way to update Python packages when updating Python from 2.6 to 2.7?

Question

I've installed A LOT of python packages for Python 2.6. Now I would like to upgrade Python to 2.7. Is there a proper or systematic way to update all the installed packages?

In my system, all the packages are installed at /usr/lib64/python2.6/site-packages/ and /usr/lib/python2.6/site-packages/

One obvious way is to install Python 2.7, download all the package sources or egg files, and re-install them one by one. However, Some useful packages like numpy and scipy are notorious for installation, especially when one needs to install from source. I expect I'll need to spend several hours to find the packages and solve the installation problems here and there.

Anyone has any suggestions on systematically update the installed packages?

Mchenry

Answer 1

First, you should not never ever ever ever install Python packages in in system library folder with easy_install using sudo on any operating system.

http://jamiecurle.co.uk/blog/installing-pip-virtualenv-and-virtualenvwrapper-on-os-x/#comment-573429347

The correct procedure would be make your installation procedure repeatable. There exist two commonly used solutions in Python world. These solutions automatically download correct versions of Python packages from http://pypi.python.org

PIP

pip and requirements.txt http://www.pip-installer.org/en/latest/requirements.html within virtualenv http://pypi.python.org/pypi/virtualenv

Buidout

Buildout, example from Plone CMS https://github.com/plone/Installers-UnifiedInstaller/blob/master/base_skeleton/versions.cfg

Buildout can also do configure, make, make install style installations for packages which need native libraries. For example there exist solution for libxml2 + lxml

http://pypi.python.org/pypi/z3c.recipe.staticlxml/

(Note: buildout does not need virtualenv as it does its own isolation from system Python)

Willettawillette

Actually, Fedora-based distros have traditionally done a very good job of separating out packages for the different versions; most of the issues are with distros that throw it all into one directory and let the various Python versions duke it out for supremacy. – Mariselamarish

It seems to me that pip cannot fully solve the original problem. Even though pip potentially solves the dependency problem and makes the installation procedure repeatable, still upgrading to 2.7 (or any newer version) would need to manually "pip install" all the packages that were installed in 2.6 (or any older version). – Mchenry

Yes, but if your requirements.txt is maintained for your application that's running one pip install command only. Or am I missing something? Of course, if you don't have dependency information only in your head then you need to reinstall all packages by hand. – Willettawillette