virtualenv doesn't copy all .py files from the lib/python directory

Question

Maybe I'm no clear on how virtualenv works, but if I run virtualenv with the --always-copy flag, I'm expecting all the files inside /lib/python to be copied.

So I create a virtual env

virtualenv -v --always-copy myenv

Output of that is

Creating myenv/lib/python2.7
Symlinking Python bootstrap modules
  Copying to myenv/lib/python2.7/config
  Copying to myenv/lib/python2.7/lib-dynload
  Copying to myenv/lib/python2.7/os.py
  Ignoring built-in bootstrap module: posix
  Copying to myenv/lib/python2.7/posixpath.py
  Cannot import bootstrap module: nt
  Copying to myenv/lib/python2.7/ntpath.py
  Copying to myenv/lib/python2.7/genericpath.py
  Copying to myenv/lib/python2.7/fnmatch.py
  Copying to myenv/lib/python2.7/locale.py
  Copying to myenv/lib/python2.7/encodings
  Copying to myenv/lib/python2.7/codecs.py
  Copying to myenv/lib/python2.7/stat.py
  Copying to myenv/lib/python2.7/UserDict.py
  File myenv/lib/python2.7/lib-dynload/readline.so already exists
  Copying to myenv/lib/python2.7/copy_reg.py
  Copying to myenv/lib/python2.7/types.py
  Copying to myenv/lib/python2.7/re.py
  Copying to myenv/lib/python2.7/sre.py
  Copying to myenv/lib/python2.7/sre_parse.py
  Copying to myenv/lib/python2.7/sre_constants.py
  Copying to myenv/lib/python2.7/sre_compile.py
  File myenv/lib/python2.7/lib-dynload/zlib.so already exists
  Copying to myenv/lib/python2.7/warnings.py
  Copying to myenv/lib/python2.7/linecache.py
  Copying to myenv/lib/python2.7/_abcoll.py
  Copying to myenv/lib/python2.7/abc.py
  Copying to myenv/lib/python2.7/_weakrefset.py
Creating myenv/lib/python2.7/site-packages
Writing myenv/lib/python2.7/site.py
Writing myenv/lib/python2.7/orig-prefix.txt
Writing myenv/lib/python2.7/no-global-site-packages.txt
Creating parent directories for myenv/include
Copying to myenv/include/python2.7
Creating myenv/bin
New python executable in myenv/bin/python
Changed mode of myenv/bin/python to 0755
Copying to myenv/bin/python2
Copying to myenv/bin/python2.7
Testing executable with myenv/bin/python -c "import sys;out=sys.stdout;getattr(out, "buffer", out).write(sys.prefix.encode("utf-8"))"
Got sys.prefix result: u'/home/rbouza/myenv'
Creating myenv/lib/python2.7/distutils
Writing myenv/lib/python2.7/distutils/__init__.py
Writing myenv/lib/python2.7/distutils/distutils.cfg
Installing setuptools, pip...
  Running command /home/rbouza/myenv/bin/python -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip
  Ignoring indexes: https://pypi.python.org/simple/
  Downloading/unpacking setuptools
  Downloading/unpacking pip
  Installing collected packages: setuptools, pip
  Successfully installed setuptools pip
  Cleaning up...
...Installing setuptools, pip...done.
Writing myenv/bin/activate
Writing myenv/bin/activate.fish
Writing myenv/bin/activate_this.py
Writing myenv/bin/activate.csh

Then I go to the lib directory

cd myenv/lib/python2.7

List all files

_abcoll.py   config        fnmatch.pyc      locale.py                    os.pyc     
    site.py            sre_parse.py   types.pyc       _weakrefset.pyc
_abcoll.pyc  copy_reg.py   genericpath.py   locale.pyc                   posixpath.py   site.pyc           sre_parse.pyc  UserDict.py
abc.py       copy_reg.pyc  genericpath.pyc  no-global-site-packages.txt  posixpath.pyc  sre_compile.py     sre.py         UserDict.pyc
abc.pyc      distutils     lib-dynload      ntpath.py                    re.py          sre_compile.pyc    stat.py        warnings.py
codecs.py    encodings     linecache.py     orig-prefix.txt              re.pyc         sre_constants.py   stat.pyc       warnings.pyc
codecs.pyc   fnmatch.py    linecache.pyc    os.py                        site-packages  sre_constants.pyc  types.py       _weakrefset.py

Which is exactly: 50 including directories, but then when I count the files/dirs in the python installation directory I get 200+ (just 1 level, not counting site-packages, etc)

Does anybody knows why is this the behavior?

Not even __future__.py is there.

Thank you.

Danadanae

I just ran into this myself, I did not expect such trickery from virtualenv! – Namhoi

I have the same issue with virtualenv from python3.4.3 :( – Feeding

Answer 1

You wouldn't expect virtualenv make unnecessary full duplicates for your python libraries.

Your Virtual Python still refers and reads library files from your root Python environment. What virtualenv does is it sets your new library install path in the virtual env, so further library installation will be confined in this virtual env.

--always-copy

This option only makes virtualenv copy necessary files rather than symlink.

Temperance

Answer 2

This is from the manual

--always-copy Always copy files rather than symlinking.

The point is that normally without this option virtualenv should only symlink the minimum required files in order to save space. This option then assures a copy of those files.

It does not say anything about copying the rest of files (esp. extra site packages you installed).

Your need to use --always-copy looks like aside of your new/extra project packages you might also want to use the Python's system site packages.

There is this option for that:

If you build with virtualenv --system-site-packages ENV, your virtual environment will inherit packages from /usr/lib/python2.7/site-packages (or wherever your global site-packages directory is).

Belongings

I find that using --system-site-packages will NOT copy the mentioned _future.py from the lib directory. – Doggoned