Unable to change Python path in reticulate

Question

The first lines I run when launching my rstudio session are:

library(reticulate)
use_python("/usr/local/lib/python3.6/site-packages")

However, when I run py_config() it shows as still using the default python 2.7 installation.

This is an issue because I'm unable to import any modules that were installed for python3. Any idea why this isn't working? I followed documentation fairly closely.

Noteworthy

I think there are some issues with startup—once a version is started, it can't be changed (or something like that). I got around it by setting it in ~/.Renviron with RETICULATE_PYTHON="/usr/local/miniconda3/bin/python" (with your desired path), though you can do the same thing with $PATH (accounting for what version of it gets used by RStudio/other editor; you may want to set that in .Renviron, too). – Perpetua

@Perpetua how do I access/find the .renviron? Newish to r so haven't used it before, it's def not obvious how to get to it. – Noteworthy

It's a text file in your home directory like .Rprofile. If you haven't used it before, you may need to create it. Here's an example. – Perpetua

Also, here's a nice overview of the startup process and the files involved. – Perpetua

Access .Renviron by running usethis::edit_r_environ(), and .Rprofile by using usethis::edit_r_profile() – Timbale

I found I had this problem after I saved my environment in RStudio. Deleting the .RData file and restarting the R session fixed the issue. – Kostroma

Answer 1

I observed that neither the technique "use_python('path')" nor the tactic of Sys.setenv(RETICULATE_PYTHON = 'path') in .RProfile worked for me (of course I am sure it must have worked for others.)

In any case the line at terminal,

which -a python python3

did produce two paths to choose from (one for python2 and one for python3 installed on my mac), so then I was able to create a ".Renviron" file in my home directory with this single line in it:

RETICULATE_PYTHON="/usr/local/bin/python3"

After I restarted RStudio, library(reticulate) activates the desired python3, and repl_python() opens a python3 interactive window, etc. etc.

Skiplane

presumably by 'home directory' you mean: /Users/<username>/ (on OSX) – Salutation

Thank you, yes, in my case it is the directory reached by the shortcut 'cd ~' or simply 'cd' – Skiplane

This was the solution after hours of searching. Can't thank you enough. – Proof

Tried others but only this solution worked for me. – Martin

You can also create a .Renviron file with the same code RETICULATE_PYTHON='Path to your python env' I also recommend setting up a virtual env with conda called reticulate where you have PyQt5 installed. – Windhoek

Answer 2

It worked for me:

Sys.setenv(RETICULATE_PYTHON = "/usr/bin/python3")
library(reticulate)

It seems important that you set RETICULATE_PYTHON before you first use reticulate.

Breadth

This works fine. On other responses, if you use several python environments in your R code, adding the path to .Renviron may be problematic. Your advice is better in such a case. – Clipped

Answer 3

use_python("path/to/python3") definitely does not work, although the Reticulate Python version configuration article says so. Don't believe it! :-)

I have tried to set the interpreter with the current Reticulate version (1.13), and the package gave me a very honest answer:

> library("reticulate")
> repl_python()
Python 2.7.15 (/usr/bin/python)
Reticulate 1.13 REPL -- A Python interpreter in R.
> use_python('/usr/bin/python3', require=T)
ERROR: The requested version of Python ('/usr/bin/python3') cannot be
used, as another version of Python ('/usr/bin/python') has already been
initialized. Please restart the R session if you need to attach
reticulate to a different version of Python.
Error in use_python("/usr/bin/python3", require = T) : 
  failed to initialize requested version of Python

Luckily, putting a .Renviron file containing the line RETICULATE_PYTHON="/path/to/python3" into the user's home directory does work:

> library("reticulate")
> py_config()
python:         /usr/bin/python3
libpython:      /usr/lib/python3.6/config-3.6m-x86_64-linux-gnu/libpython3.6.so
pythonhome:     /usr:/usr
version:        3.6.8 (default, Oct  7 2019, 12:59:55)  [GCC 8.3.0]
numpy:           [NOT FOUND]

NOTE: Python version was forced by RETICULATE_PYTHON

Finally, here comes the added value of my answer:

You can configure the Reticulate Python interpreter for all users by adding the RETICULATE_PYTHON line to the global Renviron file. It is usually found in the etc subdirectory of R's home directory. You can find out where R's home is by running the R.home() function in the R interpreter. In my case (Ubuntu 18.04.3 LTS) it was /usr/lib/R, so I edited /usr/lib/R/etc/Renviron. You obviously need admin rights to do this.

Batten

I have same problem. Can find Python path but Numpy is lost that is required for Tensorflow. How to do to get the right path for Python and to not lose Numpy ? – Repartee

Thank you so much my friend. I was searching for a fix for over 3 days now :) – Patrilineage

After reading many questions and answers on this problem, this answer worked for me! – Muniz

Answer 4

The only thing that work for me on Mac OSX, perform the following commands in the terminal:

touch $HOME/.Renviron

Then open it, I use vim, so my command is then the following:

vim $HOME/.Renviron

Add the following (for anaconda):

RETICULATE_PYTHON="/anaconda3/bin/python"

Otherwise, in the terminal type: which python3 and enter your output path

RETICULATE_PYTHON="your path from which python3"

Windhoek

thank you so much!!! It works!! I have been waiting for the solution for a year!!! – Docent

This solution also works on Ubuntu, thx a lot. Have you tried it in .Rprofile ? So the choice is dependant on each project ? – Auckland

You absolutely don’t need to touch a file before opening it with vim. It’s superfluous. – Endpaper

Answer 5

It worked for me:

• Sys.setenv(RETICULATE_PYTHON = "C:\ProgramData\Anaconda3")
• library(reticulate)
• repl_python()

Regelate

That can't possibly have worked as written, as \P is not an escape that R will recognize. You likely meant to use double backslashes. – Innards

Answer 6

You have to discover the path to your the .exe python file and then specify it in the .Renviron file.

The best is to create a conda environment to your project and then specify it as your reticulate path.

library(reticulate)

conda_create("environment name", packages = c("pandas", "matplotlib", "seaborn"), conda = "auto") # creates conda environment for your project

conda_python("environment name") # returns the .exe python path for your environment

Keep this path for the .Renviron file!!

To edit the .Renviron file:

library(usethis)

edit_r_environ()

Then, the .Renviron file will open. Paste the .exe path in it as follows:

RETICULATE_PYTHON="YOUR ENVIRONMENT PATH/python.exe"

Remember to paste it with / instead of "\".

Restart the R session and use the following command to check if everything is right:

library(reticulate)

py_config()

If everything is right, you should get something like:

python: YOUR ENVIRONMENT PATH/python.exe libpython: YOUR ENVIRONMENT PATH/python39.dll pythonhome: YOUR ENVIRONMENT PATH version: 3.9.7 (default, Sep 16 2021, 16:59:28) [MSC v.1916 64 bit (AMD64)] Architecture: 64bit numpy: YOUR ENVIRONMENT PATH\lib\site-packages\numpy numpy_version: 1.21.2

After this it should work :)

Gunmaker

Answer 7

Install numpy in the python environment you wish to use.

pip install numpy

I found that Reticulate refuses to use a version of python that does not have numpy installed.

I figured this out by running:

library("reticulate")
py_discover_config()

Reticulate skipped the first two version of python listed and used the third. I noticed it printed out the numpy version so it was probably looking for it as a requirement. It is not mentioned in the docs and should probably be added as a common problem.

Discordancy

Answer 8

For windows users, after creating a virtualenv python :

virtualenv python

this worked to use the virtualenv

Sys.setenv(RETICULATE_PYTHON = "python/Scripts/python.exe")
library(reticulate)
py_config()

EDIT: This works for any path, e.g.

Sys.setenv(RETICULATE_PYTHON = "C:/Users/UserA/Anaconda3/envs/myEnv/python.exe")

To avoid having to do it each time, add the Sys.setenv line to RProfile:

file.edit(file.path("~", ".Rprofile"))
file.edit(".Rprofile")

Hermy

Answer 9

Posting this as an answer for visibility. None of the other methods worked for me. Fortunately you can select your interpreter from Tools→Global Options→Python.

It will ask you to restart R Studio. But when you do and use reticulate::py_config() it will give the correct version.

Orang

Thanks. This worked for me in R Studio 2023.3.0 to point to right conda environment when I was stuck in a "ERROR: The requested version of Python ('C:\Users***\Anaconda3\envs\ms_py310/python.exe') cannot be used, as another version of Python ('C:/Users/****/Anaconda3/python.exe') has already been initialized." doom loop. {The forward slash in first error path looks wrong too}. – Table

Answer 10

For those looking how to solve this problem in the .onLoad.R file for their packages, this helped for me:

.onLoad <- function(libname, pkgname) {
  # create cache
  cacheEnv <<- new.env(parent = emptyenv())

  # check if r-reticulate exists in users environments
  if (!("r-reticulate" %in% reticulate::conda_list()$name)) {
    reticulate::install_miniconda()
  }

  reticulate::use_miniconda("r-reticulate")

  # instead of reticulate::use_python, 
  # set the RETICULATE_PYTHON path manually
  py_path <- reticulate::conda_python()
  Sys.setenv(RETICULATE_PYTHON = py_path)

  # check if packages need to be installed
  installed_py_packages <- reticulate::py_list_packages("r-reticulate")$package
  req_py_packages <- c("pandas", "networkx")

  for (py_package in req_py_packages) {
    if (!(py_package %in% installed_py_packages)) {
      reticulate::conda_install("r-reticulate", py_package)
    }
  }

  reticulate::configure_environment(pkgname)
}

Shroyer

Answer 11

I found a simpler solution that worked for me. By putting use_condaenv() before reticulate::py_config() it all worked as expected.

use_condaenv("scvi-env")
reticulate::py_config()
scvi <- reticulate::import(module = "scvi", convert = FALSE)

Hope this helps!

EDIT:

This meant that the version of Python found by reticulate was the one in the "scvi-env" environment rather than the default from my PATH. If the use_condaenv() call came after reticulate::py_config() then the default version of Python referenced in my PATH was used.

Kindig

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Dawdle