renv + venv + jupyterlab + IRkernel: will it blend?

Question

Short version

What is the simple and elegant way to use renv, venv and jupyterlab with IRkernel together? In particular, how to automatically activate renv from jupyter notebook that is not in the root directory?

Long version

I'm embracing a "polyglot" data science style, which means using both python and R in tandem. Now venv is awesome, and renv is awesome, and jupyterlab is awesome, so I'm trying to figure out what is the neat way to use them all together.

I almost have it, so probably a few hints would be enough to finish this setup. Here's where I'm at.

System

Start with a clean OS, and install system level requirements: R + renv and Python + venv. For example on Ubuntu it would be approximatelly like that:

# R
sudo apt install r-base
sudo R -e "install.packages('renv')"

# Python
sudo apt install python3.8
sudo apt install python3.8-venv

Project

Now create a bare bones project jupyrenv with two files:

jupyrenv/
├── DESCRIPTION
└── requirements.txt

DESCRIPTION contains R dependencies:

Suggests:
    IRkernel,
    fortunes

requirements.txt contains python dependencies:

jupyterlab

Create virtual environments and install dependencies (order matters, R has to follow python):

# Python
python3.8 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# R
R -e "renv::init(bare=TRUE)"
R -e "renv::install()"
R -e "IRkernel::installspec()"

Very neat so far!

Jupyter

launch jupyter from the command line and rejoice, it works!

jupyter-lab

What's not to like?

Unfortunatelly, if I create a folder (say notebooks) and launch an R notebook there, it does not work :(

[I 2022-02-23 19:07:24.628 ServerApp] Creating new directory in 
[I 2022-02-23 19:07:31.159 ServerApp] Creating new notebook in /notebooks
[I 2022-02-23 19:07:31.416 ServerApp] Kernel started: 0aa2c276-18dc-4511-b308-e78234fa71d4
Error in loadNamespace(name) : there is no package called ‘IRkernel’
Calls: :: ... loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart
Execution halted

Attempt to fix

It seems that renv is not used from a subfolder, so we need to hint the R process to use it. I tried to add an extra .Rprofile file the notebooks subfolder:

jupyrenv/
├── DESCRIPTION
├── requirements.txt
├── renv
├── venv
├── notebooks
│   ├── .Rprofile
│   └── Untitled.ipynb
├── .Rprofile
└── Untitled.ipynb

With the following contents:

.Rprofile:

source("../renv/activate.R")

And it kind of works, but not really. First, when trying to create an R notebook in the notebooks directory, it creates a new renv:

[I 2022-02-23 19:22:28.986 ServerApp] Creating new notebook in /notebooks
[I 2022-02-23 19:22:29.298 ServerApp] Kernel started: b40a88b3-b0bb-4839-af45-85811ec3073c
# Bootstrapping renv 0.15.2 --------------------------------------------------
* Downloading renv 0.15.2 ... OK (downloaded source)
* Installing renv 0.15.2 ... Done!
* Successfully installed and loaded renv 0.15.2.

Then that instance of jupyter works, and I can use it, but if I restart, it stops working and get's back to the missing IRkernel error:

[I 2022-02-23 19:24:58.912 ServerApp] Kernel started: 822d9372-47fd-43f5-8ac7-77895ef124dc
Error in loadNamespace(name) : there is no package called ‘IRkernel’
Calls: :: ... loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart

What am I missing?

Canalize

R essentially doesn’t cope at all with a project structure based on nested folders. Neither base R, nor the package structure nor ‘renv’ know how to handle this. They all essentially expect a completely flat structure without hierarchy. This is a major shortcoming of R. (My package ‘box’ allows creating nested folder-based projects but unfortunately this won’t help you since it doesn’t address the ‘renv’ configuration.) – Dorinda

Oh my, I was looking for something like box::use for hours! This goes into my DESCRIPTION right now 🙏 Sad to hear about the folders, though :( – Canalize

Does it have to only be on your machine? You can use JupyterLab, with R or Python kernels, or RStudio in launches from here. Choose the appropriate launch button. You can actually switch interfaces once you launch the session. There's some tricks to using the URL in the address bar. For example, from RStudio to JupyterLab, you change the end of the address from /rstudio to /lab. You can use that as a template for your repos and add what you want so you don't then need renv or venv. Binderhubs, the basis, can be deployed on your local system. – Womankind

Thanks, but I would really prefer a "barebones" solution. – Canalize

Answer 1

I opened this question as an issue in the renv github repo, and maintainers kindly provided a workaround. The contents of the notebooks/.Rprofile should be as follows:

owd <- setwd(".."); source("renv/activate.R"); setwd(owd)

It blends! 🎉

Canalize

Answer 2

I want to add my solution to this. Thanks to @psarka for pointing out, the issue is basically renv only operates in the folder it is installed in. Here is what I did, since I cannot possibly remember the exact lines to put in the .Rprofile next time I need to set this up.

ln -s /your/renv_folder/directory/ /your/notebook/directory/
ln -s /your/.Rprofile/directory/ /your/notebook/directory/

(I am on Ubuntu 22.04. There are equivalent commands on MacOS and Windows)

If you create two symlinks of .Rprofile and renv/ inside your notebook directory, wherever it is, Jupyter will correctly find IRkernel and other packages in this renv environment. The working directory is also at your notebook directory.

I should also add, I do not use venv and renv together. I use conda to do all the python things. However, the underlying cause is the same, and the solution should also work if you are using venv.

Antiscorbutic