Next with Revolution R's foreach package?

Question

I've looked through much of the documentation and done a fair amount of Googling, but can't find an answer to the following question: Is there a way to induce 'next-like' functionality in a parallel foreach loop using the foreach package?

Specifically, I'd like to do something like (this doesn't work with next but does without):

foreach(i = 1:10, .combine = "c") %dopar% {
    n <- i + floor(runif(1, 0, 9))
    if (n %% 3) {next}
    n
}

I realize I can nest my brackets, but if I want to have a few next conditions over a long loop this very quickly becomes a syntax nightmare.

Is there an easy workaround here (either next-like functionality or a different way of approaching the problem)?

Hendren

Answer 1

You could put your code in a function and call return. It's not clear from your example what you want it to do when n %% 3 so I'll return NA.

funi <- function(i) {
  n <- i + floor(runif(1, 0, 9))
  if (n %% 3) return(NA)
  n
}
foreach(i = 1:10, .combine = "c") %dopar% { funi(i) }

Olympe

Perfect -- thanks -- didn't think to use return and a function. And yes, modulus 3 is a highly arbitrary example, sorry. – Hendren

The example was fine and easily showed your situation. What I meant was that next has a different behavior than return; next means that that instance of the loop doesn't do anything at all, which is impossible when using foreach; each instance has to return something; so you'll want to make sure that you're handling whatever you choose to return appropriately. – Olympe

Answer 2

Although it seems strange, you can use a return in the body of a foreach loop, without the need for an auxiliary function (as demonstrated by @Aaron):

r <- foreach(i = 1:10, .combine='c') %dopar% {
  n <- i + floor(runif(1, 0, 9))
  if (n %% 3) return(NULL)
  n
}

A NULL is returned in this example since it is filtered out by the c function, which can be useful.

Also, although it doesn't work well for your example, the when function can take the place of next at times, and is useful for preventing the computation from taking place at all:

r <- foreach(i=1:5, .combine='c') %:%
         foreach(j=1:5, .combine='c') %:%
             when (i != j) %dopar% {
                 10 * i + j
             }

The inner expression is only evaluated 20 times, not 25. This is particularly useful with nested foreach loops, since when has access to all of the upstream iterator values.

---

Update

If you want to filter out NULLs when returning the results in a list, you need to write your own combine function. Here's a complete example that demonstrates a combine function that works like the default combine function but includes a filtering mechanism:

library(doSNOW)
cl <- makeSOCKcluster(3)
registerDoSNOW(cl)

filteredlist <- function(a, ...) {
  values <- list(...)
  c(a, values[! sapply(values, is.null)])
}

r <- foreach(i=1:200, .combine='filteredlist', .init=list(),
             .multicombine=TRUE) %dopar% {
  # filter out odd values of i
  if (i %% 2) return(NULL)
  i
}

Note that this code works correctly when there are more than 100 task results (100 is the default value of the .maxcombine option).

Anderlecht

+1. This only works for .combine='c'. If one needs it for the default list output, you have to remove the NULL list elements afterwards by r[!sapply(r, is.null)]. – Ctesiphon

@TMS I added an example that filters out NULL's when you want to return the results in a list. Although you could do it afterwards, it can be more efficient to do this in the combine function, particularly when done on-the-fly with the doSNOW and doMPI backends. – Anderlecht

Answer 3

A note to the previous answers, you should use return() or return(NULL) (like in answer 2) rather then return(NA) (answer 1). Otherwise you will get errors like below

>     error calling combine function:
>     <simpleError in rbindlist(l, use.names, fill, idcol): Item 4 has 1 columns, inconsistent with item 1 which has 16 columns. To fill
> missing columns use fill=TRUE.>

Dissonant