R kernel crashes while loading R package using rpy2

Question

First of all, I’m new to rpy2 / jupyter so please don’t judge me if this isn’t the correct place to ask my question.

I am trying to set up an integrated workflow for data analysis using R and Python and I encounter the following error:

I am on Ubuntu 19.04. running a conda environment using Jupyter 1.0.0, Python 3.7.4, R 3.5.1, r-irkernel 1.0.2 and rpy2 3.1.0 and I installed the R-package Seurat through R.

When I create a Jupyter notebook using the R-kernel, I can load Seurat with library(Seurat) just fine.

I can also use R code in python using rpy2 and the rmagic such as:

%load_ext rpy2.ipython

%%R
data(allen, package = 'scRNAseq')
adata_allen <- as(allen, 'SingleCellExperiment')

However when I try to load Seurat using rpy2 the kernel crashes:

%%R
library(Seurat)

And I get the following message:

Kernel Restarting
The kernel appears to have died. It will restart automatically

Jupyter gives the following message in the command line:

[I 16:39:01.388 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
kernel 23284ec0-63d5-4b61-9ffa-b52d19851eab restarted

Note that other libraries such as library(dplyr) load just fine using rpy2.

My complete conda environment can be found in the attached text file.

I just can’t seem to figure out what is causing the problem. Is there a way to get a more verbose error message from Jupyter?

Your help would be greatly appreciated!

Regards Felix

Earmark

This sounds like a bug in Seurat and/or (more likely) the Jupyter R kernel. It should probably be reported in their bug tracker. – Orthopedic

Answer 1

**This is an edit of the original answer - the segfault should not longer occur. **

The R package Seurat is using an other R package called reticulate, providing a bridge to Python from R.

A issue with the Python GIL being release by cffi on the rpy2 side, and the reticulate side not ensuring that the GIL was acquired caused a segfault for a long time. However, current (2023) versions of rpy2 and reticulate should work together.

Diverge

Thank you for the response! Following some of the advice from the thread you posted, I was able to fix the problem by downgrading to Seurat 3.0.2! Thanks! – Earmark

Answer 2

I had the same problem and I am also using R and python with a Jupyter notebook in docker.

I solved the Kernel crash issue by starting my notebook or Python code with this:

import os \
os.environ['R_HOME'] = '/usr/lib/R'

/usr/lib/R is where I have my system's R installation and libraries, and should be an R version needed by rpy2. Hope this helps.

Remmer

This solution inspired me to set each env variable (R_HOME, R_LIBS_USER, R_USER) manually within Windows, then restart the terminal and restart Jupyter to resolve the issue. May be a future release will set these environment variables during installation. – Argyll

Answer 3

I've got the same problem with you. But I downgrade to Seurat 3.0.2, your problem will be fixed. To use the user defined R kernel for rpy2 with conda, run the code before at the very beginning (before imoort rpy2)

# user defined R installation
import os
os.environ['R_HOME'] = '/path/to/miniconda/envs/seurat/lib/R' #path to your R installation
os.environ['R_USER'] = '/path/to/miniconda/lib/python3.7/site-packages/rpy2' #path depends on where you installed Python.

Winniewinnifred

Answer 4

This worked for me, while facing issue of kernel getting dead during importing robjects from rpy2:

import os
os.environ['R_HOME'] = '/Users/<your user>/anaconda3/envs/<env name>/lib/R'

# import your desired module
from rpy2.robjects.packages import importr

Revest

Answer 5

I tried to install rpy2 in the jupyter/r-notebook:hub-2.3.1 Docker image which comes with Python 3.10.5, IPython 8.4.0, R 4.1.3.

If I install rpy2 in a Terminal window with pip:

python3 -m pip install rpy2

and I start IPython in the Terminal, and type import rpy2,this first step works. But the next step, namely: import rpy2.robjects as robjects results in the following not-so-instructive error message:

Error in glue(.Internal(R.home()), "library", "base", "R", "base", sep = .Platform$file.sep) : 
  4 arguments passed to .Internal(paste) which requires 3
Error: could not find function "attach"
Error: object '.ArgsEnv' not found
Fatal error: unable to initialize the JIT

The reason is some subtle incompatibility between the rpy2 package on PyPI and the Python and R installations in the jupyter/r-notebook image. The incompatibility occurs because Python and R were installed using Conda in the r-notebook image.

If I install rpy2 also with Conda, like this:

conda install --yes rpy2

then everything works as advertised.

Lessons learned

1. If Python and R were installed with OS package installers, then you can probably install rpy2 with pip.
2. If Python and R were installed with Conda, then install rpy2 also with Conda.
3. (the most embarrassing bit): There is a jupyter/datascience-notebook which comes with rpy2 preinstalled (plus a lot of other goodies), no need to install anything:

jupyter/datascience-notebook includes libraries for data analysis from the Julia, Python, and R communities.

Everything in the jupyter/scipy-notebook and jupyter/r-notebook images, and their ancestor images rpy2 package The Julia compiler and base environment IJulia to support Julia code in Jupyter notebooks HDF5, Gadfly, RDatasets packages

Laudation