R: show error and warning messages in foreach %dopar%

Question

I'm new to using foreach() %dopar% for paralleling, and I have some problems about how it handles errors or warnings.

1. when I use try() with my customized error message within foreach() %dopar%, the "native" error message doesn't show up:

   test <- function(x) {
     if (x==2) "a"/2
   }
   
   foreach(i=1:3) %dopar% {
     tryout <- try(test(i))
     if (class(tryout)=="try-error") print("Error!")
   }
   

   In this case the "native" error message: Error in "a"/2 : non-numeric argument to binary operator doesn't show up, and only the Error! from try() error catching will be printed. However both error messages will be printed when not using foreach() %dopar%. So how to make both error messages show up?

2. In the above case, when there are warnings, whether additional to errors or not, the warning messages are not printed, for example with the same foreach() block as above and the test() below:

   test <- function(x) {
     if (x==2) warning("Warning!")
   }
   

   So how to show the warnings?

p.s. I found that if I simply use try(test(i)) within %dopar% then the "native" error messages and the warnings will be printed, but I do want to include my own error message in real-life situations. I also tried using tryCatch() instead of try(), but it didn't solve the problem.

Thanks!

Moccasin

You may have to tell the parallel backend you want the output (e.g. argument outfile="" of makeCluster if you are using doSNOW). There is a related question with more details. – Plumcot

Thank you for pointing to a valuable direction for solving the problem. I'm not using doSNOW for now but I'm looking into that... – Moccasin

Answer 1

I think the issue is around your errorhandling and a bit of a misunderstanding of dopar. First, think of dopar more like a function. It must return an object back to the user by default as a list (it gathers all the output from each worker and the foreach function gathers these and returns them to use, see 'output_list' below).

You can also set .errorhandling = 'pass', it will return the error back to the output object (note probably only works if .combine ='list'). Here is how .errorhandling works:

.errorhandling specifies how a task evaluation error should be handled. If the value is "stop", then execution will be stopped via the stop function if an error occurs. If the value is "remove", the result for that task will not be returned, or passed to the .combine function. If it is "pass", then the error object generated by task evaluation will be included with the rest of the results. It is assumed that the combine function (if specified) will be able to deal with the error object. The default value is "stop".

Here is an example of how to set up a tryCatch inside of a foreach. Note that errors are now stored in output_list.

output_list = foreach(i=1:3, .errorhandling='pass') %dopar% {
result <- tryCatch({
    object_that_doesnt_exist[i]}, 
     warning = function(war) {
         return('a warning')}, 
     error = function(err) {
         return('an error')}, 
     finally = {
         return('other things')
     }) # END tryCatch

  return(result)  # return your result to outputlist
}

output_list

Frentz