Python venv and virtualenvwrapper combined

Question

In Python 3.5 the recommended way to create virtual environments is with the venv, instead of virtualenv. Still the python packaging tutorial mentions both tools.

However virtualenvwrapper is a recommended wrapper tool to use when using virtualenv.

My questions are then:

1. Is there a way to use virtualenvwrapper with venv?
2. Or could one even consider virtualenvwrapper not needed due to venv? (I cannot see how this could be true since it is a wrapper solving another problem)

Edit: I can see that there is some confusion in the answers to my question. venv is Python's official equivalent of virtualenv, as explained in the links above. Multiple stack overflow questions suggests that venv should be used. As mentioned in the "duplicate" you suggested:

the introduction of venv is in part a response to that mess. If you want to help improve the situation, I suggest you use venv and encourage others to do the same

So it is encouraged to use venv. But as this questions implies, if one is to use venv how does one use a wrapper like virtualenvwrapper

Newbold

Possible duplicate of What is the difference between venv, pyvenv, pyenv, virtualenv, virtualenvwrapper, pipenv, etc? – Bottleneck

@JBentley you are entirely correct in your assumptions. I thought I was clear, but I guess not. Glad someone got it :) I'll also add that with the introduction of pipenv this issue is no longer that relevant. – Newbold

@user1853417, Could you explain your use case and workflow? I'm very curious at why pipenv in your case is an equivalent to venv + virtualenvwrapper. These tools, pipenv vs venv/virtualenv+wrapper, solve two very different problems. – Genital

Answer 1

Here is a custom but still clean and clear solution. Append This script to in .bashrc / .bash_profile / .zshrc, it will give you basic management of venv.

In addition, you can extend the script by adding the following line so that it will also display the existing venv lists.

lsvenv(){
    ls $VENV_HOME
}

Flyleaf

Doesn't answer the question. Arguably still useful information. Perhaps post as a comment, or expand this to a full answer and leave this as an alternative solution? – Threatt

I am currently using this script, and it is quite helpful to manage the virtual environments. Although I got two downvotes, I will leave it here, in case it may help some others... – Flyleaf

Answer 2

1. Is there a way to use virtualenvwrapper with venv?

Yes. Simply point WORKON_HOME at your venvs directory. Here's what I do in my ~/.zshrc, and I use a mix of virtualenv (rare now, just for a few legacy py2 needs) and venv (most common). I renamed the default to .venvs to make it clear these are mostly Python 3 venvs and not virtualenvs.

# Python Environment Handling
export WORKON_HOME=$HOME/.venvs  # Default name changed from virtualenv to highlight I am using python3 -m venv (aka pyvenv)
export PROJECT_HOME=$HOME/dev
source /usr/local/bin/virtualenvwrapper.sh  # symlinked to /Library/Frameworks/Python.framework/Versions/3.7/bin/virtualenvwrapper.sh

2. Or could one even consider virtualenvwrapper not needed due to venv? (I cannot see how this could be true since it is a wrapper solving another problem)

venv == virtualenv (in short). venv does not supersede virtualenvwrapper for the same reason virtualenv does supersede it. Your hunch on this is correct.

3. How is this better than just making an alias that sources the activate?

Well, it's up to you to determine what you need, but I find virtualenvwrapper with zsh plugins virtualenv and virtualenvwrapper to be pretty great and better than raw aliases.

workon to list all venvs is very nice, then workon speech_analyzer to jump right in on it.

4. Other solutions?

You can also set up hooks to activate venvs on directory change, but if that's what you are after and only that, then that's essentially pipenv. Pipenv is great if that's all you want to do. Pipenv also has an interesting and promising lockfile feature, but it is too slow for development and too immature with issues in production to comment on at this time.

But I've never liked the 1:1 environment per project workflow for the same reasons given here: https://chriswarrick.com/blog/2018/07/17/pipenv-promises-a-lot-delivers-very-little/. Particularly these needs line up with mine of the multi project single environment: https://chriswarrick.com/blog/2018/07/17/pipenv-promises-a-lot-delivers-very-little/#nikola

I have six environments on my machine and about 20 projects. Pipenv doesn't extend to that situation. Pipenv insists on 20 environments for 20 projects. It just doesn't work and creates more problems than it solves. If you do have a 1:1 workflow though currently, then pipenv may be the tool you'd like. One word of caution, you can ~only~ do that workflow in pipenv, unfortunately.

Genital

Answer 3

[EDIT, 8 Jul 19: Readers will probably find that this answer gives a fuller description of the different tools for handling virtual environments in Python. They all have their issues, as does conda, which has a somewhat more sophisticated concept of "environment".]

1. As its name suggests, virtualenvwrapper was specifically designed to wrap virtualenv, on which it depends. I don't believe that anyone has similarly wrapped venv yet.

2. venv is intended to do the basic work of creating virtual environments, but environment management has to be done with scripts. Although shell scripting is often people's first resort, the venv module has an extensive API to help you in those tasks.

Nowadays there are many options for creating Python virtual environments. Besides those you mention, anaconda allows the creation and management of environments and even works pretty well with pip most of the time.

The tools in the virtual environment space have been designed to work together with standard Python distribution tools as far as possible, but the arrival of venv in Python 3.5 did not invalidate either virtualenv or virtualenvwrapper, which should both still work fine.

The venv module is principally a simple toolkit to allow the creation of virtualenvs inside programs, and is by no means intended to replace the convenience of virtualenvwrapper. It simply meets a rather different set of needs.

Bottleneck

In fact this question gives a far more complete description of the issues. – Bottleneck

I'm more reluctant to downvote this answer than the other one because it is closer to being relevant, but it could do with improvement to more fully answer the question. It is not clear from this answer whether virtualenvwrapper can wrap venv, which I think is the main issue the OP is driving at (see his numbered question 1). To put it another way, is it a binary choice between (1) virtualenvwrapper + virtualenv; or (2) venv (without virtualenvwrapper)? Or is there an option (3) virtualenvwrapper + venv? – Threatt

@Threatt you are entirely correct in your assumptions. I thought I was clear, but I guess not. Glad someone got it :) I'll also add that with the introduction of pipenv this issue is no longer that relevant. – Newbold

pipenv uses virtualenv to achieve its purpose. So I think the question of virtualenvwrapper + venv still remains. But from the docs and repository of virtualenvwrapper, I don't think this is the case. – Flocky

@Newbold pipenv is pretty awful, over-marketed, and solves none of the problems venv and virtualenv do. Don't promote it as a replacement as so many do. It does one thing, deployed dependency management, and even then does it poorly. I hope the python community can standardize these things and improve the horrible packaging practices, but until then, pipenv is not the answer. Use venv or virtualenv for dev, conda/docker for deployment, and pyinstaller for shipping to grandma. chriswarrick.com/blog/2018/07/17/… – Genital