Is there something in testthat like expect_no_warnings()?

Question

I'm writing tests for a function that under some conditions will generate warnings. I want to ensure that under the other conditions it does not produce warnings. I don't see an obvious way to easily test that with testthat. I guess I could do something like:

my.result <- 25
my.func <- function() my.result
expect_equal(
  withCallingHandlers(
    my.func(), warning=function() stop("A Warning!")
  ), 
  my.result
)

or use options(warn=2), but I was hoping there would be something like:

expect_no_warnings(my.func())

Am I missing something obvious?

Ricardaricardama

I do this by setting options(warning=2) – Curr

@BenBolker, I have two issues with this approach, 1. (also a problem with the other alternate I suggested), testthat stops execution on error, instead of continuing and reporting any other failed tests; 2. I can't think of an easy systematic mechanism of undoing this that will survive non-handled errors in a test_that block. Normally I would do on.exit(options(warn=old.warn)) but to do this I'd have to wrap a function around the test_that block. Adding options(warn=old.warn) at the end wouldn't get run with the warning failure and my options would be left changed. – Ricardaricardama

Answer 1

In even more recent versions of ´testthat´ (from 0.11.0) you can do:

expect_warning(my.func(), regexp = NA)

From the documentation of expect_error

regexp: regular expression to test against. If omitted, just asserts that code produces some output, messsage, warning or error. Alternatively, you can specify NA to indicate that there should be no output, messages, warnings or errors.

So in the same way you can test that there are no messages, errors and output.

Christopher

In testthat 0.11.0 you can also use expect_silent(...) if you want to check that it emits no output, messages, or warnings all in one step. – Sidonius

@StuartR.Jefferys you are right if you want to test all of the output. Nevertheless, the question is about how to expect no warnings. So a test that prints a message will fail, even if no warning was raised. – Christopher

Absolutely. That's why I added it as a comment and listed the things it catches all in one go. The question did not say only warnings should be caught ;) I almost always use expect_silent as it is often what is really being tested and is a more restrictive. Your answer is the best way to test only for a warning and allow everything else. – Sidonius

I desire to verify no warning, however I do get an output. ie my function under test always generates a warning. Sometimes it generates a warning. I would like two test cases. One for a warning and one for output but no warning. – Vibrations

Answer 2

In recent versions of testthat, you can simply do:

expect_that(my.func(), not(gives_warning()))

Selfknowledge

As of testthat 0.10.0 (May 22 2015), the expect_that(A, func(B)) style of tests are soft deprecated, and documentation of not() states that this is to be removed in a future release. Given that, is there another answer to this other than setting options(warning=2)? – Sidonius

Sorry Karl, changed accepted answer to alko's given the deprecation. More details on github. – Ricardaricardama

actually now the optimal answer is even expect_warning(my.func(), NA), shorter and more clear – Selfknowledge

Answer 3

Update: Use expect_silent() nowadays because expect_that is deprecated, see help for the function!.

Update 2: As mentioned by @eaglefreeman the answer using expect_warning with the param regexp set to NA is the best solution since my answer causes the test to fail even if no warning was raised but just a a message was printed. This is not what the OP wanted (but just ignore warnings). I do not delete this answer to make this difference clear for other readers.

From the help's examples:

expect_silent("123")

f <- function() {
  message("Hi!")
  warning("Hey!!")
  print("OY!!!")
}

expect_silent(f())

Warning: expect_silent also expects no output so the semantics is a little bit different!

Encrust

Thanks, since this is a little more general I'm leaving the currently accepted answer as accepted (e.g. I might want to allow stdout, but not warnings for example), but this is good to know nonetheless (+1) – Ricardaricardama

I had to downvote this sorry. expect_silent is actually very misleading in this case. – Keratinize

@Keratinize Can you elaborate this please and propose a better answer to help us all to improve this? – Encrust

I used expect_silent and lost some times figuring out why it didn't work. I don't want other people to experiment the same thing :) The best answer is expect_warning with regexp set to NA. – Keratinize

@Keratinize THX a lot for your explanation, your are absolutely right! I have updates my answer to warn other readers (instead of deleting it). – Encrust

Answer 4

You can check whether expect_warning() fails. The following solves both the issue of other kind of output being produced by my.func() and not() being deprecated:

expect_failure(expect_warning(my.func()))

Chromoprotein

Answer 5

{testthat} now (as of 3.1.5, and stable since 3.2.0) exports these helpers directly:

expect_no_error(...)
expect_no_warning(...)
expect_no_message(...)

And the most general

expect_no_condition(...)

Each takes a class= argument to allow custom signal classes.

https://testthat.r-lib.org/reference/expect_no_error.html

Lupita