Running the R package reticulate in Docker environment

Question

In a previous work, I used the 'reticulate' package to run the Autogluon autoML library in R. The code works well in my current configuration (Ubuntu 20.4, R 4.10, reticulate v. 125).

However, this code doesn't work in Docker.

Dockerfile

FROM rocker/r-ver:4.1.0

## Install R packages
RUN R -q -e 'install.packages("remotes")'
RUN R -q -e 'remotes::install_github("rstudio/reticulate")'

# Install Autogluon
RUN R -q -e 'reticulate::install_miniconda()'
RUN R -q -e 'reticulate::conda_create(envname = "r-autogluon", packages = c("python=3.8.13", "numpy"))'
# RUN R -q -e 'reticulate::conda_list()'
RUN R -q -e 'reticulate::conda_install(envname = "r-autogluon", packages = "autogluon", pip = TRUE)'
RUN R -e 'reticulate::use_condaenv("r-autogluon", required = TRUE)'
# RUN -q -e 'reticulate::py_config()'

EXPOSE 3838
CMD R -e 'reticulate::import("autogluon.tabular")'

# Run in shell
# sudo docker build --no-cache -t demo .
# sudo docker run --rm -p 3838:3838 demo

I meet this error and I don't know how to solve it!

reticulate::import("autogluon.tabular") Error in py_module_import(module, convert = convert) : ModuleNotFoundError: No module named 'autogluon' Calls: -> py_module_import Execution halted

Tracks

• The 'conda_list()' indicates that the "r-autogluon" has been created with success!
• The 'py_config()' indicates that the "r-reticulate" is used by default.
• The 'reticulate::use_condaenv("r-autogluon", required = TRUE)' doesn't work.

Anybody has a solution ?

Regal

I vaguely remember that I have had problems with reticulate identifying the correct version of python even after using use_condaenv, and I had to specify with use_python('/path/to/miniconda/envs/r-autogluon/bin/python'), although that probably should have come up as a problem in py_config() – Monoplane

Answer 1

RUN R -e executes the command and closes the session : reticulate::import doesn't find autogluon.tabular because reticulate::use_condaenv('r-autogluon') ran in another session, so that the r-autogluon environment isn't set in the current session.

In order to run specific initialization commands at session launch, you need to modify Rprofile, see R startup, or ?Startup.

You could COPY a modified version of Rprofile, or modify directly the file:

FROM rocker/r-ver:4.1.0

## Install R packages
RUN R -q -e 'install.packages("remotes")'
RUN R -q -e 'remotes::install_github("rstudio/reticulate")'

# Install Autogluon
RUN R -q -e 'reticulate::install_miniconda()'
RUN R -q -e 'reticulate::conda_create(envname = "r-autogluon", packages = c("python=3.8.13", "numpy"))'
# RUN R -q -e 'reticulate::conda_list()'
RUN R -q -e 'reticulate::conda_install(envname = "r-autogluon", packages = "autogluon", pip = TRUE)'


## Modify Rprofile
RUN R -e 'write("reticulate::use_condaenv(\"r-autogluon\", required = TRUE)",file=file.path(R.home(),"etc","Rprofile.site"),append=TRUE)'
RUN R -e 'write("reticulate::import(\"autogluon.tabular\")",file=file.path(R.home(),"etc","Rprofile.site"),append=TRUE)'

EXPOSE 3838

Running the container in interactive mode now shows that autogluon.tabular is loaded (R is automatically launched because r-ver docker file ends with CMD ["R"]):

docker run --rm -ti demo

R version 4.1.0 (2021-05-18) -- "Camp Pontanezen"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

Module(autogluon.tabular)
>  
 

Sleazy

This is a very insightful answer. Curious if there's a way to write the RUN R -q -e commands to execute multiple lines of R code at once with semicolons so they're executed in the same session. Or to run an R script (as alternatives to editing Rprofile). – Agribusiness

You can put two R commands together separated by ;, for example RUN R -e 'write("reticulate::use_condaenv(\"r-autogluon\", required = TRUE)",file=file.path(R.home(),"etc","Rprofile.site"),append=TRUE);write("reticulate::import(\"autogluon.tabular\")",file=file.path(R.home(),"etc","Rprofile.site"),append=TRUE)' works – Sleazy

You can also source() any R script. I modified RProfile in order to be able to open an interactive session, but this isn't needed for a batch session. – Sleazy

Sorry, I didn't word my question well. I'm asking if running multiple commands with semicolon or sourcing an R script could be ways to solve this problem without having to edit Rprofile. – Agribusiness