Python: select one of multiple installed module versions

Question

On my system, I have several modules installed multiple times. To give an example, numpy 1.6.1 is installed in the standard path at /usr/lib/python2.7/dist-packages, and I have an updated version of numpy 1.8.0 installed at /local/python/lib/python2.7/site-packages/.

The reason I cannot simply remove the old version is that I do not have permissions to change anything on my work computer. I however need to use the new numpy version.

I have added /local/python/lib/python2.7/site-packages/ to my PYTHONPATH. Unfortunately, this does not help, since /usr/lib/python2.7/dist-packages is inserted into the path first and therefore, numpy 1.6.1 will be loaded. Here's an example:

>>> import os
>>> print os.environ['PYTHONPATH']
/local/python/lib/python2.7/site-packages
>>> import pprint
>>> import sys
>>> pprint.pprint(sys.path)
['',
 '/local/python/lib/python2.7/site-packages/matplotlib-1.3.1-py2.7-linux-x86_64.egg',
 '/local/python/lib/python2.7/site-packages/pyparsing-2.0.1-py2.7.egg',
 '~/.local/lib/python2.7/site-packages/setuptools-3.4.4-py2.7.egg',
 '~/.local/lib/python2.7/site-packages/mpldatacursor-0.5_dev-py2.7.egg',
 '/usr/lib/python2.7/dist-packages',
 '/local/python/lib/python2.7/site-packages',
 '/usr/lib/python2.7',
 ...,
 '~/.local/lib/python2.7/dist-packages', 
 ...]

So, it seems that the import order is

1. current directory
2. eggs from PYTHONPATH
3. eggs from local module path (~/.local/lib/python2.7/site-packages/*.egg)
4. system-wide module path (~/usr/lib/python2.7/dist-packages/)
5. directories from PYTHONPATH
6. intermediate paths (omitted for brevity)
7. userbase directory (~/.local/lib/python2.7/site-packages/)

My problem is that I would need to put item 5. before items 3. and 4. for my code to work properly. Right now, if I import a module that was compiled against numpy 1.8.0 from the /local/* directory, and this module imports numpy, it will still take numpy from the /usr/* directory and fail.

I have circumvented this problem by placing something like this in my scripts:

import sys
sys.path.insert(0, '/local/python/lib/python2.7/site-packages/')

Thereby I can force Python to use the right import order, but of course this is not a solution, since I would have to do this in every single script.

Mock

The real solution to your problem is your own copy of Python' s modules. – Braddock

I don't really see how this would help, because the modules from the userbase are imported after the modules of the ones from the global base. I updated my question for clarity. – Mock

I should have said your own copy of Python. – Braddock

I agree - but if I had permissions to install anything here, I would simply update the modules and thus solve the problem. The point is, I do not have the permissions, it's a work computer, and I have to circumvent the problem with the multiple module versions. – Mock

You can install your own copy of Python in your home directory, you don't need to install it system-wide. – Braddock

you could use a usercustomize.pyto force-reorder or prune the paths across all of your own code, assuming you can get a file into the site-packages directory of your base install. Check the docs for the site module docs.python.org/2/library/site.html – Selfpity

Answer 1

Besides the suggestions already given in the comment section, have you thought about using virtualenv? This would give you fine-grained control over every module that you want to use. If you're not familiar with virtualenv you'll want to read the documentation to get a feel for how it works.

Purely for example, you could install and set it up, like so (virtualenv-1.11.6 looks to be the most recent version currently):

$ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.11.6.tar.gz
$ tar xvfz virtualenv-1.11.6.tar.gz
$ cd virtualenv-1.11.6
$ python virtualenv.py ../numpyvenv
$ cd ../numpyvenv
$ source ./bin/activate
(numpyvenv) $ pip install numpy
# downloads, compiles, and installs numpy into the virtual environemnt
(numpyvenv) $ python
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.version.version
'1.9.1'
>>> quit()
(numpyvenv) $ deactivate
$ # the virtual environment has been deactivated

Above, we created a virtual environment named "numpyvenv", activated the environment, installed numpy, printed the numpy version (to show it works), quit python, and deactivated the environment. Next time you activate the environment, numpy will be there along with whatever other modules you install. You may run into hiccups while trying this, but it should get you started.

Pancake

Answer 2

I had this problem on a Mac I was using without administrator access. My solution was the following:

1. Find the directory of the numpy version you want to use. For me this was /Library/Python/2.7/site-packages

2. Create a file ~/.startup.py and point to it with PYTHONSTARTUP=~/.startup.py in your .bashrc file

3. In .startup.py:

import sys

sys.path.insert(0,'/Library/Python/2.7/site-packages/') <--- imports this BEFORE the standard parts

import numpy

print("Importing numpy version"+numpy.__version__) <---- To remind that we have changed the numpy version

This seems to work fine for me. I hope it helps.

Disordered

Had not read the original question properly so realised that part of my solution is already suggested in the original question, apologies for that – Disordered

Answer 3

While a virtualenv seems the way to go, as of Force python to use an older version of module (than what I have installed now) you can also use a modification of

import pkg_resources
pkg_resources.require("Twisted==8.2.0")
import twisted

Sanitary

Answer 4

I had the same issue on Debian Wheezy after installing the latest numpy module with easy_install.

The new numpy module was installed in /usr/local/lib/python2.7/dist-packages/numpy while the old module was in /usr/lib/pymodules/python2.7/numpy. When I tried to import the numpy module, the older version was imported. And as you say, adding to PYTHONPATH the new module path does not help, because is added in the sys.path below the older entry.

The issue seem to be in easy-install, because it creates a file easy-install.pth that imports /usr/lib/pymodules/python2.7 before any local module.

To fix the issue I just edited the file /usr/local/lib/python2.7/dist-packages/easy-install.pth and commented out the line /usr/lib/pymodules/python2.7 so this line will be placed below in the sys.path.

Hesperus