How can I make R's output more verbose so as to reassure me that it hasn't broken yet?

Question

I often run code that eats up a lot of RAM, and may take as much as an hour before it gives its outputs. Often, I'll be half an hour in to running such code and I'll be worrying that something gone wrong. Is there any way that I can get R to reassure me that there's not been any errors yet? I suppose that I could put milestones in to the code itself, but I'm wondering if there's anything in R (or RStudio) that can automatically do this job at run time. For example, it would be handy to see how much memory the code is using, because then I'd be reassured that it's still working whenever I see the memory use significantly vary.

Rabbi

you could add a progress bar as a indicator . Besides you could use, stopifnot() statements , that stop the running loop if something went wrong . – Delossantos

"Using memory" is not a safe (or good, IMO) measure that things are still progressing; for example, R could be consuming more memory due to properly expanding a matrix or frame ... or it could be exploding memory and about to crash OOM. The only way that I know of to know that your code is still progressing it to put in some form of manual indicator, whether with something like utils::txtProgressBar (or similar packages), or some logging such as message(...) or the logger package. – Mooncalf

I'm personally a fan of the logger package, injecting several logger::log_debug or logger::log_trace throughout the expensive areas so I know things are progressing, and logger::log_info in top-level places. – Mooncalf

Is it just a single R command that eats-up most of the execution time and is called only once (eg. a complex optimization)? Then it will become difficult to log the progress... – Incomer

@RYoda Sometimes it is, but usually it isn't. It's often a single user-written function that has many functions within it. – Rabbi

@Mooncalf Why not post that as an answer? – Rabbi

Primarily, I guess, because "logging" and "indication of progress" are slightly different. Another technique I use is add message(".",appendLF=FALSE), which gives a no-CRLF string of . across the screen as the loop progresses. For fast-moving, I'll often use a counter such as i <- i+1; if (i%%100 == 0) message(".",appendLF=TRUE). But in my head it seems more of a hack than your question asking "if there's anything in R that can automatically do this job". That also does nothing to indicate memory use, something that I suspect only profiling is going to support (currently) in R. – Mooncalf

Answer 1

You might like my package {boomer}.

If you rig() your function, all its calls will be exploded and printed as the code is executed.

For instance

# remotes::install_github("moodymudskipper/boomer")
fun <- function(x) {
  x <- x + 1
  Sys.sleep(3)
  x + 1
}

library(boomer)

# rig() the function and all the calls will be exploded
# and displayed as they're run
rig(fun)(2)

Schaffel

Answer 2

One way is:

1. to make a standalone file containing all the stuff to be run,
2. sourcing it and getting warned when the code is done, possibly with error.

The small function warn_me below:

• runs the source file located in "path"
• possibly catches an error, if an error there was
• plays a sound when the run is over
• sends an email reporting the status of the run: OK or fail
• optionally: plays a sound until you stop it, so you can't miss it's over

And here it is:

warn_me = function(path, annoying = FALSE){
    # The run
    info = try(source(path))
    
    # Sound telling it's over
    library(beepr)
    beep()
    
    # Send an email with status
    library(mailR)
    msg = if(inherits(info, "try-error")) "The run failed" else "It's over, all went well"
    
    send.mail(from = "[email protected]",
              to = "[email protected]",
              subject = msg,
              body = "All is in the title.",
              smtp = list(host.name = "smtp.mailtrap.io", port = 25,
                          user.name = "********",
                          passwd = "******", ssl = TRUE),
              authenticate = TRUE,
              send = TRUE)
    
    if(annoying){
        while(TRUE){
            beepr::beep()
            Sys.sleep(1)
        }
    }
}

warn_me(path)

I didn't test the package mailR myself, but any email sending package would do. See this excellent page on sending emails in R for alternatives.

Hailey

Answer 3

If you are running an R script file within RStudio, use the "Source with Echo" selection (Ctrl+Shift+Enter, or via dropdown).

Alodi

I don't see why this would be expected to make any difference. Do you have a certain type of code in mind? – Rabbi