Replicate virtualenv without downloading all the packages again on the same machine

Question

I have a couple projects that require similar dependencies, and I don't want to have pip going out and DLing the dependencies from the web every time. For instance I am using the norel-django package which would conflict with my standard django (rdbms version) if I installed it system wide.

Is there a way for me to "reuse" the downloaded dependancies using pip? Do I need to DL the source tar.bz2 files and make a folder structure similar to that of a pip archive or something? Any assistance would be appreciated.

Thanks

Rambort

Answer 1

Add the following to $HOME/.pip/pip.conf:

[global]
download_cache = ~/.pip/cache

This tells pip to cache downloads in ~/.pip/cache so it won't need to go out and download them again next time.

Grind

It tells pip to cache downloads in ~/.pip/cache so it won't need to go out and download them again next time. Apologies for the terse answer; it seems kinda self-explanatory to me (but then maybe I haven't had enough coffee yet). – Grind

Actually a quick google search looks like this does what i need. – Rambort

Answer 2

it looks like virtualenv has a virtualenv-clone command, or perhaps virtualenvwrapper does?

Regardless, it looks to be a little more involved then just copyin and pasting virtual environment directories:

https://github.com/edwardgeorge/virtualenv-clone

additionally it appears virtualenv has a flag that will facilitate in moving your virtualenv.

http://www.virtualenv.org/en/latest/#making-environments-relocatable

$ virtualenv --relocatable ENV from virtualenv doc:

This will make some of the files created by setuptools or distribute use relative paths, and will change all the scripts to use activate_this.py instead of using the location of the Python interpreter to select the environment.

Note: you must run this after you’ve installed any packages into the environment. If you make an environment relocatable, then install a new package, you must run virtualenv --relocatable again.

Also, this does not make your packages cross-platform. You can move the directory around, but it can only be used on other similar computers. Some known environmental differences that can cause incompatibilities: a different version of Python, when one platform uses UCS2 for its internal unicode representation and another uses UCS4 (a compile-time option), obvious platform changes like Windows vs. Linux, or Intel vs. ARM, and if you have libraries that bind to C libraries on the system, if those C libraries are located somewhere different (either different versions, or a different filesystem layout).

If you use this flag to create an environment, currently, the --system-site-packages option will be implied.

Damales

I don't need to move the site-packages folder so much as just have a duplicate of what is installed in there installed in my new venv. I know I could do a pip freeze and import into my new venv but that still requires me to go out on the web, which for something like a dev box seems like it could be cumbersome and slow since I know I already have the packages on the machine somewhere. – Rambort

@PaulJ.Warner, i was trying to provide some suggestions on ways you could clone your virtualenv so that packages would not be reinstalled – Damales

Answer 3

Try symlinking the site-packages folders.

Manifestative