Using R package source files in packrat (rather than CRAN) with Travis-CI

Question

I'm working with an R package that is an RStudio project, and I'm using packrat to keep a local copy of the source of the packages I depend on for my project.

I've got Travis-CI checking my R package each time I make a commit, but each time Travis builds my package it gets the latest version of the dependent packages, rather than the versions I've got in my packrat/ directory.

I can see in richfitz/wood that he appears to have achieved this goal with this in his .travis.yml file:

env:
 USE_PACKRAT=1

and a fairly complex make/packrat.mk file which makes it all work.

My question is what is the simplest way to configure my project (e.g. my .travis.yml file) to tell the Travis machine to get the packages from my packrat/ directory on github, and not from CRAN?

Lucilelucilia

Answer 1

After much trial and error and further reading, it seems that this will do it, with a .travis.yml file like this:

# R for travis: see documentation at https://docs.travis-ci.com/user/languages/r

language: R
sudo: false
cache: packages
install:
  - R -e "0" --args --bootstrap-packrat
warnings_are_errors: false

The key lines in the above file are:

install:
  - R -e "0" --args --bootstrap-packrat

This will start R, and builds the R packages in the local packrat directory so that they are available in the Travis machine.

After that, travis will continue and attempt to build the package, and will not need contact CRAN to get the dependencies because they are already available (assuming packrat is working as expected).

I discovered this trick here: https://travis-ci.org/ChowHub/paper-pattern-similarity/builds/127262823 and at https://github.com/rstudio/packrat/issues/158. I've got it working here: https://travis-ci.org/benmarwick/mjbtramp/builds/157747326

The advantage of this is we can build on travis with the exact same packages that we're using locally. We don't have to get the latest packages from CRAN when we build on travis, now we can have more control of the package versions that travis builds with in our project.

The disadvantage is that the build time on travis is substantially increased. One of my projects went from 2-3 mins to 13-15 mins after switching to packrat.

---

UPDATE After Noam's question below and Jim's comment, it seems we can cache the packrat packages using cache: like this:

# R for travis: see documentation at https://docs.travis-ci.com/user/languages/r

language: R
sudo: false
cache:
  directories: $TRAVIS_BUILD_DIR/packrat/
  packages: true
install:
  - R -e "0" --args --bootstrap-packrat
warnings_are_errors: false

In my use-case, this has reduced the times substantially, back to 1-2 mins.

Lucilelucilia

Does this long build time persist? Or are the packages cached after the first build? – Shel

Good question, yes, seems that the times are steady and high when using packrat. Here are my use-case build summaries: travis-ci.org/benmarwick/mjbtramp/builds, the one build where I turned off packrat specifically to see the timing comes in at just under 2 mins, with the rest, using packrat, 13-18 mins. Caching packages built from packrat would probably help! – Lucilelucilia

Caching is possible! I've updated my answer to show how – Lucilelucilia

just a small caveat: the above caching method, which greatly reduces build times (yeah!) does not invalidate automatically, when the packrat dependencies (and therefore packrat.lock) change. So, whenever you add / update / remove a package from packrat, Travis will not know about this, as long as caching is on. You have manually delete the cache and build again to fix this snafu. I've raised an issue – Miracle

Ongoing work on this problem is also documented here: github.com/rstudio/packrat/issues/352 (thanks @maxheld!) – Lucilelucilia