Can I automatically generate unit tests for testthat from roxygen2 examples?

Question

I'm working on the PKNCA package for R. While developing the testing code, some of the tests would also be good examples. I want to keep them as both (test and example). Is there a way that I can embed something within the roxygen2 documentation that will get copied into the testing, too?

What I'm thinking about is documentation like:

#' @exampleTest
#' set.seed(5)
#' rnorm(1) ## -0.8409

And that would generate a test like:

expect_equal({set.seed(5)
              rnorm(1)}, -0.8409, tol=1e-4)

(The tol came from the fact that it is a number and the number of digits shown in the example.)

Bein

Answer 1

Use devtools::run_examples() as explained in the check chapter of Hadley Wickham's book on packages. Function examples are tested when you run R CMD CHECK. This is not part of testthat but rather of the standard R package checking system.

Shiflett

Reading the documentation, this looks like it will confirm that the examples ran without errors, but it doesn't look like it will perform accuracy testing. I guess I could run my examples where they require accuracy, but that doesn't put it directly into the testing framework. If there aren't other answers forthcoming, I'll set this as the answer. But, I'm hoping for a way to confirm accuracy of the examples, too. – Bein

More directly, testthat::test_examples will run the examples in package during testing. The tests pass when there are no errors when running the examples. – Gummous

Answer 2

There is a way, but it's not as smooth as you'd like. You'll have to call testthat functions inside your @examples block. Here's an example function:

#' @examples
#'   testStrings <- c("1234567890",
#'                    "123 456 7890")
#'
#'   testthat::expect_equal(extractPhoneNumbers(testStrings), "0123")
extractPhoneNumbers <- function(inputStr) {
    # check input:
    if (!is.character(inputStr)) {
        stop("'inputStr' must be a (vector of) string(s)!")
    }

    # imports
    `%>%` <- stringr::`%>%`
    replace_all <- stringr::str_replace_all
    extract_all <- stringr::str_extract_all

    # intermediary regex's
    visualDelimitersRegex <- "[()+\\-_. ]"
    phoneNumberRegex <- "[:digit:]{10}"

    inputStr %>%
    replace_all(pattern = visualDelimitersRegex, replacement = "") %>%
    extract_all(pattern = phoneNumberRegex)
}

When you run devtools::run_examples() or devtools::check, both will throw errors since the call to testthat::expect_equal() throws an error.

Example output from devtools::check looks like

*** SNIP ***
* checking for unstated dependencies in examples ... OK
* checking examples ... ERROR
Running examples in ‘demoPkg-Ex.R’ failed
The error most likely occurred in:

> base::assign(".ptime", proc.time(), pos = "CheckExEnv")
> ### Name: extractPhoneNumbers
> ### Title: Extract Phone Numbers
> ### Aliases: extractPhoneNumbers
> 
> ### ** Examples
> 
>   testStrings <- c("1234567890",
+                    "123 456 7890")
> 
>   testthat::expect_equal(extractPhoneNumbers(testStrings), "0123")
Error: extractPhoneNumbers(testStrings) not equal to "0123"
Modes: list, character
Length mismatch: comparison on first 1 components
Component 1: 1 string mismatch
Execution halted
* checking for unstated dependencies in ‘tests’ ... OK
* checking tests ...
  Running ‘testthat.R’
 OK
* checking PDF version of manual ... OK
* DONE

Status: 1 ERROR

Artificial

thanks for the suggestion. As you suggested, it's not as smooth as I'd like. Since will obfuscate the example I don't think I'll use it, but I have suggested that using testthat with examples could help. – Bein