Issues in R package after CRAN asked to replace \dontrun{} by \donttest{}

Question

I submitted a package to CRAN and they asked me to replace \dontrun{} by \donttest{} in the Rd-files and resubmit. I was using \dontrun{} to wrap some examples that are supposed to throw error messages.

After replacing \dontrun{} by \donttest{} I conducted some tests and R CMD check still succeeds but it happens that now both devtools::check() and R CMD check --as-cran fail due to the examples wrapped in \donttest{}:

checking examples with --run-donttest ... ERROR

After some browsing I learned that R 4.0.0 has changed R CMD check --as-cran to run \donttest examples. According to the NEWS of R-devel:

"R CMD check --as-cran now runs \donttest examples (which are run by example()) instead of instructing the tester to do so. This can be temporarily circumvented during development by setting environment variable R_CHECK_DONTTEST_EXAMPLES to a false value."

Since I intend to resubmit the package to CRAN, setting _R_CHECK_DONTTEST_EXAMPLES_ to false locally will not help me.

I also found this recent discussion in a devtools issue where Hadley Wickham states that:

"Generally, now if you don't want to run tests on CRAN \dontrun{} is more likely to work, but using \dontrun{} may cause initial submission to fail."

So now I don't know how to proceed because if I resubmit the package with the required changes I already know it will throw an error in R CMD check --as-cran, and hence it will probably fail CRAN's automatic pretests.

EDIT:

As suggested here I tried if(interactive()){} instead of \dontrun{}. This solution succeeds in R CMD check --as-cran and devtools::check() but I don't think it is the most appropriate way to address this problem since it does not work well with example() (throws an error and does not show the remaining examples). \dontrun{} works better with example() as it prints all the examples but comments out the ones wrapped with \dontrun{}.

Lemniscus

I’m voting to close this question because it's more appropriate for the [email protected] mailing list ... it's an administrative question rather than a programming question ... – Rhythm

"I was using \dontrun{} to wrap some examples that are supposed to throw error messages." - maybe just remove these from the examples and instead put these to testthat for unit tests? Or are these examples that throw errors needed for the users to understand the package? – Neapolitan

@SteffenMoritz removing these exemples would indeed fix the errors in R CMD check. But in this package I have some functions that stop the execution with informative error messages depending on certain conditions and I think it would be useful (even thought not essential) to illustrate this. – Lemniscus

Posted an answer :) What I quite often have already seen is, that people add things to the examples, but comment it out. Might be a good compromise. – Neapolitan

@SteffenMoritz comment out the examples is definitely a good possibility and I will probably end up doing that if nothing better comes up. I'm also considering if(interactive()){} instead of \donttest{}, as suggested here here – Lemniscus

Interesting, might be a good idea. Did not use this before. I guess the downside (in comparison to \donttest{} ) is, that this would be code the user actually sees in the example. Which could be somehow confusing. Or am I wrong here? – Neapolitan

@SteffenMoritz you are right, the documentation will have if(interactive()) in the lines of the exemples that i'm trying not to run, and in that sense \donttest{} would result in "tidier" documentation since \donttest{} is not printed. But I don't think \donttest{} is an option here since it will return actual errors when running R CMD check --as-cran (and also when running example(). But this is still an improvement when compared to \dontrun because \dontrun is also printed to the documentation. – Lemniscus

@SteffenMoritz Also, if(interactive()){} allows the user to run examples line by line since the condition evaluates to TRUE. \donttest{} is printed to the documentation and so the user has to select only the code that is inside the {}, which is not so user friendly – Lemniscus

Answer 1

If you know that something will throw an error, you can wrap it in try().

## example of failing code
try(stop("Here is an error"))

Giotto

Thank you for your answer. This seems to be the best solution so far. It succeeds in R CMD check --as-cran and properly displays the error messages in example() without stopping the execution. Im still considering my options but if this ends up being the actual solution I will come back to accept this answer. – Lemniscus

I can now confirm that CRAN accepted this solution. – Lemniscus

Answer 2

I don't think the package examples are the right place for "examples that are supposed to throw error messages".

Your problem would be easily solved when you move these 'examples' to testthat unit tests.

There is

expect_error()
expect_warning()

to see if your package throws a warning/error as expected.

If you really want to inform users about what they should avoid to input, maybe you can just add it as a comment to the examples or into the other documentation (details, param)

What you see quite regularly in other package examples is the following:

## Example for working
function(x, abc = "5)

## This would give an error because
# function(x, abc = "falsch")

## Working example 2
function(x)
x <- x+y

Neapolitan

Thank you for your answer! I'm already using unit tests to make sure the error messages are as expected (among other things). The error messages I mentioned are not about informing users about what kind of input should be avoided. Imagine an helper_foo(x, y) that you use inside foo(x, y) to stop the execution of foo() in case some conditions regarding x and y are TRUE and inform the user which of the conditions triggered the error. helper_foo(x, y) is a function and not just code embedded in foo() because it can be used in many other functions with similar input structure. – Lemniscus