How to specify (non-R) library path for dynamic library loading in R?

Question

I keep getting the following error when attempting to install readxl or haven in R (both dependencies of tidyverse) post-compilation, when the installer runs the loading test:

** testing if installed package can be loaded
Error in dyn.load(file, DLLpath = DLLpath, ...) :
  unable to load shared object '<my_lib_Path>/readxl/libs/readxl.so':
  <my_lib_path>/readxl/libs/readxl.so: undefined symbol: libiconv
Error loading failed

I have libiconv.so in a local lib path (not for R packages) that is included in LD_LIBRARY_PATH and I've verified in my R session that Sys.getenv("LD_LIBRARY_PATH") has that directory. Why can't R's dynamic library loader find this shared object? Is there a different R-specific environment variable I need to define to have the dynamic library loader in R search my local lib path?

Please note that this is not an issue with an R library path, but instead for a non-R dependency that an R package has. If I were compiling and linking C++ code, gcc would use ld, and hence LD_LIBRARY_PATH to track down dynamic dependencies. R doesn't appear to respect this rather common approach, and I can't seem to find any documentation on how to manage these more fine-grained dependency issues.

---

Additional Details

!> sessionInfo()
 R version 3.3.3 (2017-03-06)
 Platform: x86_64-pc-linux-gnu (64-bit)
 Running under: CentOS Linux 7 (Core)

 locale:
  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
  [9] LC_ADDRESS=C               LC_TELEPHONE=C
 [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

 attached base packages:
 [1] stats     graphics  grDevices utils     datasets  methods   base
 > 

I had previously compiled libiconv because it was a dependency for something else (don't recall what now - likely not an R package given current problems). I tried reinstalling it, but made no difference.

---

Edit

I have also tried manually loading the library prior to installation:

> dyn.load(".local/lib/libiconv.so")
> is.loaded("libiconv")
[1] TRUE
> install.packages("tidyverse")

but it fails just as above.

Crimmer

Have you tried deleting readxl or haven from the /library/ folder and re-installing? – Wernick

I am not sure if this is helpful, but perhaps you should check the output of .libPaths()? You can also use it to "gets/sets the library trees within which packages are looked for." – Vitebsk

@Wernick By default, R cleans up (deletes) libraries that fail to fully install, so invoking install.packages("tidyverse") always attempts to install readxl and haven afresh. – Crimmer

@Vitebsk The libiconv library is not an R package, and therefore is not located in the R library directory(s). However, I did try making a symlink to libiconv.so in my local R library folder to see if the loader would find it there: it doesn't. – Crimmer

What's the output of ldd /<my_lib_path>/readxl/libs/readxl.so , executed from within R ? Maybe that's informative. – Ariosto

@Ariosto Even though R does successfully build a readxl.so, since the package fails the loading test, the installer proceeds to delete everything, so there is no readxl.so that I can work with. – Crimmer

@Ariosto once I built from source outside R, I was able to retain the src/readxl.so - even though it didn't install - and your suggestion ended up being very informative. Thanks for the help! – Crimmer

@Crimmer glad to hear that. I've often relied on ldd to check installation/loading problems. Often it's not directly helpful, but "indirectly". – Ariosto

Answer 1

Normally, the iconv method is picked up from glibc, which is linked to during build of the R packages in question. For whatever reason, however, iconv is getting resolved to libiconv in this case, but it is not linked by the R packages during build.

Original Workaround

One can make the linking to libiconv explicit by adding the following line to the haven/src/Makevars source file

PKG_LIBS=-liconv

which then let's you install from source R CMD INSTALL haven. However, editing packages feels hacky, plus this is something that will need to be done every upgrade, which sounds like a hassle.

Cleaner Workaround

Another option is to use withr::with_makevars, which allows one to temporarily control Makevars content. With this technique, one can install directly from the repo:

withr::with_makevars(c(PKG_LIBS="-liconv"), install.packages("haven"), assignment="+=")

---

Credit: @knb suggested that I inspect the readxl.so with ldd and this turned out to be super useful because it showed that the shared object wasn't even trying to link to libiconv. Knowing that, I realized I could manually add the reference via the -liconv flag. Thanks @knb!

Additional Info

On the package side of things, relevant details about connecting libraries to R packages can be found in the guide for building libraries. On the system configuration side, the R-admin guide has some useful sections.

Crimmer

This is a valid approach, especially when install.packages("haven") (part of tidyverse system which depends upon libiconv) fails to load after successful compilation. In some cases (including mine), setting LD_LIBRARY_PATH or ~/.Renviron or <R_HOME>/etc/Renviron.site to include path to libiconv.so did not resolve error of Error in dyn.load. – Mallorca

Answer 2

Are you running the code in RStudio Server? If so, the answer here may be useful.

I used to meet a similar error while loading dynamic library. The library was in a path contained in LD_LIBRARY_PATH. When I ran the code in R console, it could load the dynamic library correctly. But when I ran it in RStudio, the same error in your post raised.

The reason is that RStudio Server has its own library search path environment. You should specify the following configuration in /etc/rstudio/rserver.conf:

rsession-ld-library-path=/usr/lib64/:/usr/local/lib/:OTHER_PATH_OF_YOUR_LIB

Restart RStudio Server and the error should be fixed.

Cyril

Good to know, but no RStudio in my case. For now just R via ESS. – Crimmer

This was very helpful! It was an rstudio-server issue in my case! Thanks! – Hollo

I'd similar issue but turns out recompiling package, e.g., haven with PKG_LIBS flag (similar to merv's reply) solved the issue for RStudio server though I ran install command using bash terminal from a separate ssh session and not using Rstudio console or embedded terminal. That way, I can force R to use user configured LD_LIBRARY_PATH over one set by rstudio server. – Mallorca

Answer 3

These libraries really should be standard on RH-based system too, and be found.

If you must add them to R, you have to do so before you start R. One way is via LD_LIBRARY_PATH, a better way is to edit a file in /etc/ld.so.conf.d/ (assuming RH/CentOS have that too). Else maybe via /etc/environment.

Edit: If /etc/ is out of reach, you can do everything below $HOME. Standard shell instantiation works, and R has its own .Rprofile and .Renviron. You can have those below $HOME for all your projects, and/or in a per-project directory---see help(Startup).

Witless

Yes, you're right that they are typically standard; this is an atypical situation. As for editing stuff in /etc/..., I am on a shared HPC cluster, so I lack permissions to edit there. – Crimmer

Why do I need to add the libraries before I start R? Why can't I just edit the LD_LIBRARY_PATH using Sys.setenv? – Junitajunius

@GregorSturm it is not possible on linux to effectively change LD_LIBRARY_PATH of a running process. – Bette

Answer 4

I put the export LD_LIBRARY_PATH=... statements into a file ~/.profile. This way, both command line R and RStudio Server are able to find the shared library.

In my case, I was trying to get the Rglpk package to locate the libglpk.so file. According to Stefaan Lippen's blog entry, the .profile file is the preferred location for these type of configurations not strictly related to bash.

Monadnock

Good to know, but in this particular case, changing LD_LIBRARY_PATH was insufficient to achieve successful compilation. – Crimmer

Answer 5

Did you install R via rpm or compile it yourself?

solution 1

If you have the permission to modify the R executable (shell script), you can try this:

Edit ~/.local/bin/R or /usr/local/bin/R or /usr/bin/R

#!/bin/bash
# Shell wrapper for R executable.

export LD_LIBRARY_PATH="<my_lib_Path>/readxl/libs/"
R_HOME_DIR=...
...
...

Solution 2

Or you may vim ~/.local/bin/R

#!/bin/bash
# Shell wrapper for R executable.

export LD_LIBRARY_PATH="<my_lib_Path>/readxl/libs/"

/usr/bin/R

then add ~/.local/bin to your PATH

Coats

As per OP, I already have LD_LIBRARY_PATH defined (via .bashrc) and I have verified that R correctly sees this environment variable on all sessions. The problem is the installing libraries do not tell g++ to link to libiconv. – Crimmer

Since Ubuntu 17.04, LD_LIBRARY_PATH can't be defined in .bashrc. I didn't know the case of CentOS. But maybe everything will work after you define LD_LIBRARY_PATH in R shell script. – Coats