Installing anaconda with pyenv, unable to configure virtual environment

Question

I am using pyenv to manage python installations and virtual environments - and I would like anaconda to be one such installation, and to be able to create virtual environments using anaconda python. Using pyenv install anaconda3-2019.03 successfully installs and I can activate the version with pyenv global anaconda3-2019.03:

SamLee-PC:~ max$ pyenv versions
  system
  2.7.10
  2.7.10/envs/flask_tutorial
* 3.7.4 (set by /Users/max/.pyenv/version)
  3.7.4/envs/learning_python
  3.7.4/envs/microblog
  3.7.4/envs/stocktool
  3.7.4/envs/test1
  anaconda3-2019.03
  anaconda3-2019.03/envs/datsci
  datsci
  flask_tutorial
  learning_python
  microblog
  stocktool
  test1
SamLee-PC:~ max$ pyenv which python
/Users/max/.pyenv/versions/3.7.4/bin/python
SamLee-PC:~ max$ pyenv global anaconda3-2019.03
(anaconda3-2019.03) SamLee-PC:~ max$ pyenv which python
/Users/max/.pyenv/versions/anaconda3-2019.03/bin/python
(anaconda3-2019.03) SamLee-PC:~ max$ pyenv which conda
/Users/max/.pyenv/versions/anaconda3-2019.03/bin/conda

(I do not know why the python version is added to the command prompt only with the anaconda installation)

When I creat a new virtual environment (pyenv virtualenv anaconda3-2019.03 datsci), it appears to work as intended:

SamLee-PC:~ max$ pyenv which python
/Users/max/.pyenv/versions/3.7.4/bin/python
SamLee-PC:~ max$ cd code/linkedin/datsci
(datsci) SamLee-PC:datsci max$ pyenv which python
/Users/max/.pyenv/versions/datsci/bin/python
(datsci) SamLee-PC:datsci max$ 

This is the same relative location returned returned by pyenv which python in my other virtual environments, which all work properly.

The problem is that the python version that gets run in the virtual environment doesn't have access to all of anaconda's packages:

(datsci) SamLee-PC:datsci max$ anaconda-navigator
pyenv: anaconda-navigator: command not found

The `anaconda-navigator' command exists in these Python versions:
  anaconda3-2019.03

(datsci) SamLee-PC:datsci max$ 

though it does if I activate the python installation manually:

SamLee-PC:~ max$ pyenv global anaconda3-2019.03
(anaconda3-2019.03) SamLee-PC:~ max$ anaconda-navigator
WARNING: The conda.compat module is deprecated and will be removed in a future release.
/Users/max/.pyenv/versions/anaconda3-2019.03/lib/python3.7/site-packages/anaconda_navigator/api/conda_api.py:1364: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  data = yaml.load(f)
2019-09-21 14:03:38,666 - ERROR download_api._download:234
Invalid url https://www.anaconda.com/wp-content/uploads/2017/05/Webinar20-20Three20Ways20to20Move20your20Data20Science20Projects20to20Production.png
#anaconda navigator GUI opens

(oddly, the python version that datsci points to does seem to recognize conda:

(datsci) SamLee-PC:datsci max$ pyenv which python
/Users/max/.pyenv/versions/datsci/bin/python
(datsci) SamLee-PC:datsci max$ conda
usage: conda [-h] [-V] command ...

conda is a tool for managing and deploying applications, environments and packages.

...

The problem is that, as I understand it, activating the python version this way whenever I want anaconda means that I only have access to one version of anaconda python - defeating the purpose of using it in a virtual environment.

One more piece of the puzzle - I noticed that for my other virtual environments, ~/.pyenv/versions/name-of-virtualenv contains a file pyvenv.cfg, which contains e.g. the following:

  1 home = /Users/max/.pyenv/versions/3.7.4/bin                                                                   
  2 include-system-site-packages = false
  3 version = 3.7.4

There was no such file in ~/.pyenv/versions/datsci, so I added one:

  1 home = /Users/max/.pyenv/versions/anaconda3-2019.03/bin                                                       
  2 include-system-site-packages = false
  3 version = anaconda3-2019.03

to no avail.

FWIW the contents of that directory are:

(anaconda3-2019.03) SamLee-PC:datsci max$ pwd
/Users/max/.pyenv/versions/datsci
(anaconda3-2019.03) SamLee-PC:datsci max$ ls
bin     conda-meta  include     lib     pyvenv.cfg  share       ssl
(anaconda3-2019.03) SamLee-PC:datsci max$ 

I wonder if I have the wrong contents in the pyvenv.cfg file for datsci...but I am at a loss.

I really like the way pyenv works, and I would like to continue using it. Is there a way to configure anaconda to behave properly under pyenv?

Update:

I uninstalled my pyenv-managed anaconda and did a fresh install as per Simba's recommendation. Now by default pyenv determines the active python installation, and I can activate conda with conda activate base:

Maxs-MacBook-Air:~ max$ which python
/Users/max/.pyenv/shims/python
Maxs-MacBook-Air:~ max$ conda activate base
(base) Maxs-MacBook-Air:~ max$ which python
/Users/max/anaconda3/bin/python
(base) Maxs-MacBook-Air:~ max$ 

However after creating a new conda environment with conda create --name datsci, the new conda environment does not seem to activate properly:

Maxs-MacBook-Air:~ max$ conda activate datsci
(datsci) Maxs-MacBook-Air:~ max$ which python
/Users/max/.pyenv/shims/python

What am I missing?

Here are the contents of my .bash_profile:

  1 export PATH="/Users/max/.pyenv/bin:$PATH"                                                                                                                                                                                                 
  2 eval "$(pyenv init -)"
  3 eval "$(pyenv virtualenv-init -)"
  4 
  5 # >>> conda initialize >>>
  6 # !! Contents within this block are managed by 'conda init' !!
  7 __conda_setup="$('/Users/max/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
  8 if [ $? -eq 0 ]; then
  9     eval "$__conda_setup"
 10 else
 11     if [ -f "/Users/max/anaconda3/etc/profile.d/conda.sh" ]; then
 12         . "/Users/max/anaconda3/etc/profile.d/conda.sh"
 13     else
 14         export PATH="/Users/max/anaconda3/bin:$PATH"
 15     fi
 16 fi
 17 unset __conda_setup
 18 # <<< conda initialize <<<

Is it important that I duplicate the if statement found in Simba's .bash_profile?

Update 2:

Problem solved, I had to install python in the new environment.

Litterbug

Answer 1

Personal recommendation: Don't use pyenv to install Anaconda or Miniconda.

Both pyenv and conda are able to manage different python environments. The anaconda installed by pyenv should only serves as a Python interpreter. Python environment creation from anaconda installed by pyenv is still handled by pyenv virtualenv but not conda env create.

I've been using these two tools together. It turns out the best solution is to install conda, pyenv separately, and manage their virtual environments separately as well.

1. alway initialize pyenv
2. only expose command conda but don't activate any environment

Detail

Install pyenv.

Install Anaconda or Miniconda normally, NOT by pyenv install.

Make sure the pyenv and conda commands are available in an interactive shell.

Initialize pyenv by putting following content into shell init file (.bashrc for Bash, .zshrc for ZSH).

# Put the content into ~/.bashrc or ~/.bash_profile for Bash,
# .zshrc for ZSH

# you may need to add dir of command `pyenv` into PATH,
# if command pyenv is not available yet

if command -v pyenv &>/dev/null; then
    eval "$(pyenv init -)"
fi
if command -v pyenv-virtualenv &>/dev/null; then
    eval "$(pyenv virtualenv-init -)"
fi

Expose command conda but don't activate any environment, even the base environment. Execute the following commands in your shell.

# Run the content in the shell

# init conda, the following command write scripts into your shell init file automatically
conda init

# disable init of env "base"
conda config --set auto_activate_base false

Note: After this setup, the default python is the one set by pyenv global. Use pyenv and conda to manage environments separately.

Examples of managing virtual environments.

# virtual environments from pyenv
pyenv install 3.6.9
pyenv virtualenv 3.6.9 new-env
pyenv activate new-env
pyenv deactive
# You can also use `pyenv local`


# virtual environments from conda
conda env create new-env python=3.6
conda env list
conda activate new-env
conda deactivate

Default env location for pyenv is ~/.pyenv/versions.

Default env location for conda, check output from conda info.

References

• pyenv installation
• pyenv virtualenv installatoin
• How do I prevent Conda from activating the base environment by default?

Elsieelsinore

thanks for this! conda config --set auto_activate_base false did the trick for me. – Kimberleekimberley

the second if should probably look for the command pyenv virtualenv (no hyphens) – Selda

@SaverioGuzzo You misunderstood it. pyenv virtualenv is not one command, it's command pyenv calling its sub command virtualenv. pyenv virtualenv foo calls pyenv and pass virtualenv foo as arguments to it. pyenv accepts these args, dispatches the calling to command/script pyenv-virtualenv, and pass foo as an argument. Basically, pyenv-virtualenv foo will be called at last. That's how pyenv works. – Elsieelsinore

pyenv-virtualenv advertises (on their GitHub page github.com/pyenv/pyenv-virtualenv) that it can manage conda environments. why do you prefer to keep them separate? – Anathema

Thanks for the detailed instructions. Otherwise quite some time should be spent to figure out how to run pyenv and conda in a non-conflicting seamless way on the same machine. – Sayette

Why should the base environment not be activated? – Howsoever

Thank you so much for this detailed explanation and recipe, I was managing miniconda with pyenv and it was a nightmare – Tisatisane

Thanks for your post @Simba. Would you elaborate more on installing conda separately? What are the pros and cons? Some experts advise installing it with pyenv. For example: Use pyenv and virtual environments to manage Python complexity. As Wouter van Amsterdam indicated pyenv-virtualenv shows examples where conda was installed through pyenv. Thanks – Ostracism