Why does virtualenv inherit $PYTHONPATH from my shell?

Question

So I'm migrating all my tools from python2 to python3.4 on an Ubuntu 14.04 machine. So far I've done the following:

1. aliased python to python3 in my zshrc for just my user
2. installed pip3 on the system itself (but I'll just be using virtualenvs for everything anyway so I won't really use it)
3. changed my virtualenvwrapper "make" alias to mkvirtualenv --python=/usr/bin/python3 ('workon' is invoked below as 'v')

Now curiously, and you can clearly see it below, running python3 from a virtualenv activated environment still inherits my $PYTHONPATH which is still setup for all my python2 paths. This wreaks havoc when installing/running programs in my virtualenv because the python3 paths show up AFTER the old python2 paths, so python2 modules are imported first in my programs. Nulling my $PYTHONPATH to '' before starting the virtualenv fixes this and my programs start as expected. But my questions are:

1. Is this inheritance of $PYTHONPATH in virtualenvs normal? Doesn't that defeat the entire purpose?
2. Why set $PYTHONPATH as an env-var in the shell when python already handles it's own paths internally?
3. Am I using $PYTHONPATH correctly? Should I just be setting it in my 'zshrc' to only list my personal additions ($HOME/dev) and not the redundant '/usr/local/lib/' locations?
4. I can very easily export an alternate python3 path for use with my virtualenvs just before invoking them, and reset them when done, but is this the best way to fix this?

    ○ echo $PYTHONPATH
    /usr/local/lib/python2.7/site-packages:/usr/local/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages:/home/brian/dev

    brian@zeus:~/.virtualenvs
    ○ python2
    Python 2.7.6 (default, Mar 22 2014, 22:59:56)
    [GCC 4.8.2] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import sys, pprint
    >>> pprint.pprint(sys.path)
    ['',
     '/usr/local/lib/python2.7/dist-packages/pudb-2013.3.4-py2.7.egg',
     '/usr/local/lib/python2.7/dist-packages/Pygments-1.6-py2.7.egg',
     '/usr/local/lib/python2.7/dist-packages/urwid-1.1.1-py2.7-linux-x86_64.egg',
     '/usr/local/lib/python2.7/dist-packages/pythoscope-0.4.3-py2.7.egg',
     '/usr/local/lib/python2.7/site-packages',
     '/usr/local/lib/python2.7/dist-packages',
     '/usr/lib/python2.7/dist-packages',
     '/home/brian/dev',
     '/usr/lib/python2.7',
     '/usr/lib/python2.7/plat-x86_64-linux-gnu',
     '/usr/lib/python2.7/lib-tk',
     '/usr/lib/python2.7/lib-old',
     '/usr/lib/python2.7/lib-dynload',
     '/usr/lib/python2.7/dist-packages/PILcompat',
     '/usr/lib/python2.7/dist-packages/gst-0.10',
     '/usr/lib/python2.7/dist-packages/gtk-2.0',
     '/usr/lib/pymodules/python2.7',
     '/usr/lib/python2.7/dist-packages/ubuntu-sso-client',
     '/usr/lib/python2.7/dist-packages/ubuntuone-client',
     '/usr/lib/python2.7/dist-packages/ubuntuone-storage-protocol',
     '/usr/lib/python2.7/dist-packages/wx-2.8-gtk2-unicode']
    >>>

    brian@zeus:~/.virtualenvs
    ○ v py3venv
    (py3venv)
    brian@zeus:~/.virtualenvs
    ○ python3
    Python 3.4.0 (default, Apr 11 2014, 13:05:11)
    [GCC 4.8.2] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import sys, pprint
    >>> pprint.pprint(sys.path)
    ['',
     '/usr/local/lib/python2.7/site-packages',
     '/usr/local/lib/python2.7/dist-packages',
     '/usr/lib/python2.7/dist-packages',
     '/home/brian/dev',
     '/home/brian/.virtualenvs/py3venv/lib/python3.4',
     '/home/brian/.virtualenvs/py3venv/lib/python3.4/plat-x86_64-linux-gnu',
     '/home/brian/.virtualenvs/py3venv/lib/python3.4/lib-dynload',
     '/usr/lib/python3.4',
     '/usr/lib/python3.4/plat-x86_64-linux-gnu',
     '/home/brian/.virtualenvs/py3venv/lib/python3.4/site-packages']
    >>>
    (py3venv)

Goree

Answer 1

I stumbled onto this answer about $PYTHONPATH which solved this for me just now. Essentially, setting $PYTHONPATH is optional and is a convenience to the user. It should only contain additional paths the user wants to add to their python path so that the user doesn't have to do this in python itself just to run a script from the terminal.

So to solve my problem above, I set my $PYTHONPATH (in my zshrc) to only my additional folder of '$HOME/dev' and nothing else. This eliminated the references to python2 in my path and all my python3 programs are starting as expected in my virtualenv.

Goree

Answer 2

You can also change the Pythonpath by adding to your virtualenv's /bin/activate file:

export PYTHONPATH="/your/path"

Further explained here:

Quote:

To have it restored to its original value on deactivate, you could add

export OLD_PYTHONPATH="$PYTHONPATH"

before the previously mentioned line, and add the following line to your bin/postdeactivate script.

---

You might also want to have a look at this answer which talks about using add2virtualenv for adding directories.

Insert

Answer 3

The $PYTHONPATH appears in your virtualenv because that virtualenv is just a part of your shell environment, and you (somewhere) told your shell to export the value of PYTHONPATH to child shells.

One of the joys of working in virtual environments is that there is much less need to put additional directories on your PYTHONPATH, but it appears as though you have unwittingly been treating it as a global (for all shells) setting, when it's more suited to being a per-project setting.

EDIT 2 Oct, 2023: Since it's never recommended to modify your system Python in any way on a developer machine*, learning to use virtual environments makes it easy to keep the dependencies for your different projects separate, and so avoid conflicts among them.

Since 3.7 you can create a virtual environment with the command python3.x -m venv pathname. Once created, you can activate it with source pathname/bin/activate (on most shells - alternatives are provided for others), and deactivate it with deactivate - or just terminate the shell process. Once you've done with it, simply rm -rpathname to delete it.

Don't be intimidated like I was - I wasted a year before I started using them!

* You never know what your system's relying on it for!

Campanology