Debugging 'testthat' tests in RStudio

Question

Is it possible to invoke the debugger in RStudio when running testthat tests? I haven't been able to find a setup that allows this (various combinations of "use devtools package functions if available" in the settings, hitting the "Test Package" option in the "Build -> More" menu, running test() in the console, putting in browser() calls, etc.) but haven't found a way yet.

I also find myself getting lost a lot when testing, unsure whether the code being run has been installed in the system libraries (by doing 'Build & Reload'), or is being run in situ from the local R directory, or what - sometimes RStudio complains that a breakpoint can't be set until the package is rebuilt (so I suspect the former) or doesn't (so I suspect the latter). Not sure if this issue is closely related or not to my main question.

Without finding a way to drop into the debugger, I end up pasting test code into the console & working in a very ad-hoc fashion, and basically shooting my TDD habits in the foot. So any advice would be appreciated - if it's not possible to invoke the debugger, any suggested workarounds?

I'm running RStudio version 0.99.447 on OS X, in local mode, with R 3.2.1.

Edit - I'd also love to know more background about the options, e.g. "option X will never support debugging, because it's running in a forked process, try this other option Y instead."

Update - having had no responses here, I also asked at https://support.rstudio.com/hc/communities/public/questions/204779797-Debugging-testthat-tests-in-RStudio (where I also haven't had any responses).

Metathesis

Answer 1

The following works for me:

1. Insert a call to browser() somewhere within the testthat unit tests.
2. Run devtools::test() from the RStudio console (instead of using the "Test Package" menu item from the UI)

Then, when the test runner hits the browser() invocation, you should be able to use the environment browser and step through the code.

I haven't found a way to get testthat to stop at breakpoints, but inserting browser() invocations is a pretty close substitute.

To be absolutely sure that you're starting from a consistent state when it comes to loading packages, you can close RStudio, re-open it, run "Clean and rebuild", and then devtools::test()

Lastly, if you're working within an RStudio package, you may want to check out the following advice from RStudio support:

In order to debug effectively in your package, you’ll also want to ensure that your package is compiled with the --with-keep.source option. This option is the default for new packages in RStudio; if you need to set it manually, it can be found in Tools -> Project Options -> Build Tools.

Corrosion

By "Clean and rebuild" do you mean "Build & Reload"? The thing I really hate about that is that I often accidentally install my development version globally on the machine by doing it. dev_mode is one solution but sometimes I forget to turn that on too, since it's a manual process... – Metathesis

I'm hesitant to use the browser() method either - recently someone on my team (not me, thankfully!) left a bunch of browser() statements in their code, which went to production, which was bad. It's a bummer to have to alter the code just to scrutinize it. Although it's not quite as bad in this case to do it in the test files - for some reason when using devtools::test() breakpoints don't work in the test files, but they do work in the code files under the R/ directory. – Metathesis

No, there's a separate "Clean and rebuild" command, which as far as I can tell is the same as "Build & reload" but performs some initial cleanup to get into a more predictable state. – Corrosion

Also, based on the following discussions, it looks like the lack of support for breakpoints in testthat is a known issue, but not yet fixed as of this writing: discussion 1, discussion 2 – Corrosion

Cool, thanks for the pointer. I've added a "poke" on github.com/hadley/testthat/pull/117 . – Metathesis

Coming back to an old thread - I never use "Clean and Rebuild" or "Build and Reload" anymore. I only use "Load All", which is quick and doesn't install globally. "Document" is also necessary sometimes if imports need to be adjusted. – Metathesis

I have tried following you suggestions with browser() but testthat still does not stop at breakpoints, --with-keep.source is set. Tried re-opening RStudio and running "Clean and rebuild" to no avail. Do you know by any chance what else it may be? – Laburnum

Error in loadNamespace(name) : there is no package called ‘devtools’ – Gomez

I got non-interactive browser() -- left over from debugging? – Tanga

Answer 2

If you want to use editor breakpoints from RStudio in order to debug unit tests, you can do it by working around testthat. As Hadley said in a Github issue, testthat usually runs in a separate session from RStudio, so it won't ever be able to use editor breakpoints. However, the tests, themselves, are function calls with two arguments, a description and code. You can read these and run them with a little code.

testf_trace <- function(filename, match) {
  env <- new.env()

  test_that <- function(desc, code) {
    if (length(grep(match, desc)) > 0) {
      eval(substitute(code), env)
    }
  }
  env$test_that <- test_that
  source(filename, env)
}

If you have a test that reads test_that("invariant is true", { f(3) == 6 }), then you can test this by running testf_trace(filename, "is true").

If you're unsure where a function is defined, and hence whether the breakpoint will work, you can find out by typing the name of the function at the console. The last line or two will tell you in which environment the function is defined, if it was defined in an environment other than .Global.

Aeneas