Using different versions of python with virtualenvwrapper

Question

I've got various versions of python installed on my Mac using Macports. When I've selected python 2.7 via $ port select python python27, virtualenvwrapper works perfectly.

But if I select another version of python, i.e. 2.6, virtualenvwrapper generates an error message: ImportError: No module named virtualenvwrapper.hook_loader

I checked my .profile and it's setting VIRTUALENVWRAPPER_PYTHON to /opt/local/bin/python, so it seems to me virtualenvwrapper should work regardless of which python I've selected.

Any idea what would cause virtualenvwrapper to generate a .hook_loader error when I switch python versions?

Bork

Without going through port select ... and sticking with your base 2.7, does just running mkvirtualenv --python /path/to/python2.6 work? It should automatically switch to (and set up the environment with) the correct interpreter. On my system (set up with homebrew), mkvirtualenv -p python2.6 works fine. – Leflore

I don't get the hook_loader error, but it's complaining about lack of DEST_DIR $ mkvirtualenv --python /opt/local/bin/python2.7 Running virtualenv with interpreter /opt/local/bin/python2.7 You must provide a DEST_DIR – Bork

Whoops, sorry--left out the key argument! That should be mkvirtualenv --python /path/to/python2.6 env_name. mkvirtualenv makes a folder called "env_name" in your $WORKON_HOME, which gets passed on to virtualenv as its DEST_DIR argument. Without specifying a name, it would have a hard time figuring out where to set things up, that's for sure. – Leflore

Duh. I should have caught that. Yes, that worked. Guess the answer is to leave port select to python27 and run mkvirtualenv with the --python flag when I need to use something else. – Bork

Answer 1

I know this is pretty much solved in your comments, but it's mac only,

and even more I think the correct way should be to set VIRTUALENVWRAPPER_PYTHON to the real python you are using on the command line.

To be sure you can do which python.

Actually, you can even do:

export VIRTUALENVWRAPPER_PYTHON=`which python`

On linux I do this in my .bashrc, so all in all, assuming you installed virtualenv and created your first "virtual environment" virtualenv (how original)

. virtualenv/bin/activate
export WORKON_HOME=$HOME/.virtualenvs # or whatever else you want
export VIRTUALENVWRAPPER_PYTHON=`which python`
export PROJECT_HOME=SOMETHING
source $HOME/virtualenv/bin/virtualenvwrapper.sh # or wherever else you got that installed

(and by the way, you wrote:

I checked my .profile and it's setting VIRTUALENVWRAPPER_PYTHON to /opt/local/bin/python, so it seems to me virtualenvwrapper should work regardless of which python I've selected

which is actually the opposite - virtualenv relies on using the correct python (and the packages that go with it) so it's very important to set the python path accordingly.

Even running a py file with a "#!/bin/python" might bring trouble once you are virtualenved!

Austreng

Running a few tests on my Mac -- it appears VIRTUALENVWRAPPER_PYTHON only controls which Python executable is used by virtualenvwrapper itself, not the executable that's installed into the virtual environment e.g. when you run mkproject. I would love to be wrong but so far it appears the only way to choose a different Python version is to use -p / --python in the virtualenvwrapper command line. If that's true, it's kind of a shame. – Morez

@ChrisJohnson mmmh since then I stopped using virtualenvwrapper - not finding much need for it anymore - I have no easy way to double check quickly, but you might indeed be right... Also on MAC I use brew now... – Austreng

Same as @ChrisJohnson on ubuntu. had my VIRTUALENVWRAPPER_PYTHON set to python2, but mkvirtualenv was creating virtualenvs with python3. – Misogyny

Answer 2

You can select the python version explicitly

mkvirtualenv -p python3 venvname

or

mkvirtualenv -p python2.7 venvname

Toxic

Perfect for being able to easily run multiple different environements on a single development machine. – Foredo

How can I be sure that the virtual environment created has the correct python installed? If I do which python I get: /home/USER/.virtualenvs/venvname/bin/python – Pursuant

@Pursuant type workon venvname and python3 --version, this should show the python3 version being used – Executant

Worked on Windows as well. Perfect. – Kamat

Answer 3

None of these worked. I installed Python3 first when setting up my osx machine, and pip and all default to that.

First, check which python you have installed:

$ `which python` -V

If this returns "Python 2.7.12", then you are set to run:

$ mkvirtualenv -p `which python` api_server
Running virtualenv with interpreter /usr/local/bin/python
New python executable in /Users/eric/.virtualenvs/api_server/bin/python2.7
Also creating executable in /Users/eric/.virtualenvs/api_server/bin/python
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating /Users/eric/.virtualenvs/api_server/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/eric/.virtualenvs/api_server/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/eric/.virtualenvs/api_server/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/eric/.virtualenvs/api_server/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/eric/.virtualenvs/api_server/bin/get_env_details

This will also activate the api_server workon, which changes your python executable:

$ which python
/Users/eric/.virtualenvs/api_server/bin/python
$ python -V
Python 2.7.12

What does which python actually do? It outputs the directory of the python executables found in your PATH:

$ which python
/usr/local/bin/python

By using which python, you are basically passing in /usr/local/bin/python to the -p option in the mkvirtualenv directory.

What happens when you have more than one python executable returned in which python? Just find the one you want and pass it in:

$ mkvirtualenv -p /usr/local/bin/python3 api_server

And virtualenvwrapper will end up using that python executable instead.

Hairsplitting

Answer 4

Confirmed the use of two similarly-named environment variables:

VIRTUALENVWRAPPER_PYTHON -- which Python version is used by the virtualenvwrapper utility itself. In other words, which version of Python executes virtualenvwrapper, as if that Python version had been explicitly named in the #! line of the virtualenvwrapper script file.

VIRTUALENV_PYTHON -- which Python version will be installed by virtualenv when you create a new virtual environment. Equivalent to -p / --python option on the virtualenv command line.

And perhaps obviously :) the version of Python run in a virtual environment is the version you install for that environment -- has no relation to the above environment variables after the env is created.

See https://mcmap.net/q/25077/-upgrade-python-in-a-virtual-environment for how to upgrade Python within a virtualenv.

Morez

Answer 5

To use multiple versions of python with virtualwrapper you need to reinstall the virtualwrapper for that new version with its corresponding pip manager.

Say you just installed Python-3.12.0 then in your terminal you will need to

sudo pip3.12 install virtualenv virtualenvwrapper

then you can proceed to create your environment with the different python version

mkvirtualenv sexyspace -p python3.12
workon sexyspace
python

Note that just my typing python it will load the correct python version. Also you can pip install normally within the environment since it knows that it has to use pip3.12 internally

Eroto

Answer 6

You (the OP) seem to have installed virtualenv and virtualenvwrapper with python2.7, and not with python2.6. If python2.6 is called at the moment your shell loads the virtualenvwrapper.sh script, it is unhappy. Pretty straightforward.

VIRTUALENVWRAPPER_PYTHON is made for those situations. With it, you can make sure you always use the right version of python, and don't have to always add that -p /path/to/python2.7

So, I don't agree with Stefano's answer in that case, in the OP's situation, you should have explained clearly in your .bashrc which python to use:

...
export VIRTUALENVWRAPPER_PYTHON=/path/to/your/python2.7
source /path/to/bin/virtualenvwrapper.sh

Like that it should be ok all the time! Virtualenvwrapper is done to simplify things.

Also, please note that /opt/local/bin/python must be a symlink to the version of python you select with port python select (check that with ls -l /opt/local/bin/python to be sure).

Gregson

I'd like to stress that using the -p flag is a solution if you've got layered terminals that prevent you from setting an environment variable (like I have). mkvirtualenv -p /usr/bin/python3 Foo – Abrade