How to write unit tests for suggested packages?

Question

Packages in R can have different types of dependencies on other packages. Some of these types indicate hard requirements, i.e. Depends, Imports and LinkingTo.

However, there is a second category that indicate a softer dependency, i.e. Suggests and Enhances. In both these cases, the package provides additional functionality if the suggested / enhanced package is available.

Here is a concrete example: The package checkpoint imports knitr because knitr helps checkpoint to parse rmarkdown files.

But now I am considering changing knitr to a Suggests dependency, i.e. only provide this functionality if knitr is actually installed.

For proper unit testing, this means I have to test both scenarios:

1. If knitr is available, then do stuff.
2. If knitr is not available, then throw a warning and do nothing.

The actual R code is simple:

if(require(knitr)) {
  do_stuff()
} else {
  message("blah")
}

Question

But how can I set up unit tests for both scenarios?

The way I see it, the simple fact of checking for require(knitr) will load the knitr package if it is available in the local library.

So, to test for case 1, I have to install knitr locally, meaning I can't test for case 2.

Is there a way of configuring testthat (or any other unit testing framework) for this use case?

Mucoviscidosis

Would temporarily renaming the knitr folder be an acceptable solution? – Dragrope

@Dragrope Maybe, and this might work on my local machine. But how do I manage this process on my continuous build integration machine (Jenkins or Travis) and how do I write this test to work on CRAN? – Mucoviscidosis

I would imagine that if you install knitr on those machines it goes into a folder where you have read/write privileges. I suppose it would boil down to detecting if/where knitr is installed to by using dir on the entries in .libPaths() and then using file.rename to rename the knitr folder to something like knitr_BALEETED (or whatever you want) and then renaming it back to knitr after the tests are done. I'm not sure how CRAN compliant this would be... – Dragrope

It's an interesting idea. I'll try using system.file() to tell me the location. To be honest, I'd be happy to disable the test on CRAN (easy to do with testthat) conditional on the test working locally and on Travis. – Mucoviscidosis

I didn't realize testthat had an option to skip_on_cran. That... actually makes some of the stuff I've been working on easier. – Dragrope

Could use find.package("knitr", quiet=TRUE) to find an installed knitr, or length(find.package("knitr", quiet=TRUE)) as a test for its presence on .libPaths(). – Irfan

There's also the new skip_if_not_installed(), but that won't help here. I'd make the function being tested switch behaviours based on an explicit argument, e.g. knitr_installed = system.file(package="knitr") != "". Then you test could more easily evaluate both cases – Lanielanier

This would only work with some setup ahead of time, and would fail on a standard installation (so should probably be used in conjunction with skip_on_cran), but you can put the suggested packages in a separate library and as part of the test setup make sure that library is/is not on the search path to test the two different cases. – Dogooder

Answer 1

tl;dr

To test the branch followed when use require(knitr) fails, use trace() to temporarily modify require() so that it won't find knitr, even if it is present on .libPaths(). Specifically, in the body of require(), reset the value of lib.loc= to point to R.home() -- an existing directory that does not contain a knitr package.

This seems to work just as well in a package as it will in an interactive session in which you run the following:

find.package("knitr")

trace("require", quote(lib.loc <- R.home()), at=1)
isTRUE(suppressMessages(suppressWarnings(require(knitr))))

untrace("require")
isTRUE(suppressMessages(suppressWarnings(require(knitr))))

---

As I understand this, you have a function with two branches, one to be performed in R sessions for which require(knitr) succeeds, and the other to be performed in sessions where it fails. You are then wanting to test this function "both ways" from a single R instance in which knitr actually is on .libPaths().

So basically you are needing some way to temporarily blind the call require(knitr) to the actual presence of knitr. Completely and temporarily resetting the value returned by .libPaths() looked promising, but doesn't seem to be possible.

Another promising avenue is to somehow reset the default value of lib.loc in calls to require() from NULL (which means "use the value of .libPaths()) to some other location where knitr is not available. You can't accomplish this by overwriting base::require(), nor (in a package) can you get there by defining a local masking version of require() with the desired value of lib.loc.

It does, though, look like you can get away with using trace() to temporarily modify require() (blinding it to knitr's availability by setting lib.loc=R.home()). Then do untrace() to restore require() to the vanilla version which will go ahead and find knitr.

Here's what that looked like in the dummy package I tested this with. First an R function that allows us to test for success along the two branches

## $PKG_SRC/R/hello.R

hello <- function(x=1) {
    if(require(knitr)) {
        x==2
    } else {
        x==3
    }
}

Then a couple of tests, one for each branch:

## $PKG_SRC/inst/tests/testme.R

## Test the second branch, run when require(knitr) fails
trace("require", quote(lib.loc <- R.home()), at=1)
stopifnot(hello(3))
untrace("require")

## Test the first branch, run when require(knitr) succeeds
stopifnot(hello(2))

To test this, I used pkgKitten::kitten("dummy") to set up a source directory, copied in these two files, added Suggests: knitr to the DESCRIPTION file, and then ran devtools::install() and devtools::check() from the appropriate directory. The package installs just fine, and passes all of the checks.

Headquarters

Even better might be to set trace("require", quote(value <- FALSE), at=5). That will cause require() to return a value of FALSE even if knitr is already loaded, allowing you to pay less attention to the order in which you perform tests. – Irfan

This is very nice. I have been working on something similar, using the testthat::with_mock() feature. I suspect under the hood with_mock does something similar. – Mucoviscidosis

@Mucoviscidosis Very cool, I hadn't seen that before. trace() will be nicer when you just want to change one or a few lines of a function, but with_mock() sure makes it easy to simply force a certain return value, which is all you need here. – Irfan