How to use doMC under Windows or alternative parallel processing implementation for glmnet?

Question

I am on Win7 OS with R 3.3.1 in Rstudio. Intention is to use glmnet with parallel processing. From the ?glmnet help:

parallel: If TRUE, use parallel foreach to fit each fold. Must register parallel before hand, such as doMC or others. See the example below.

From the referenced example:

# Parallel
require(doMC)
registerDoMC(cores=4)

install.packages('doMC') returns package is not available. Manually checking CRAN gives downloadable UNIX code but Windows binaries are not available.

Can I still use doMC like code under my Win7 OS or what is a useful alternative?

Rozanna

Answer 1

As written in the vignette to doMC

The doMC package acts as an interface between foreach and the multicore functionality of the parallel package, originally written by Simon Urbanek and incorporated into parallel for R2.14.0. The multicore functionality currently only works with operating systems that support the fork system call (which means that Windows isn't supported)

You can try to use the snow package and a SOCK cluster instead. (Thx @HongOoi for the hint that loading doSNOW is not really required.)

library(doParallel)

#the following line will create a local 4-node snow cluster
workers = makeCluster(4, type="SOCK")
registerDoParallel(workers)

foreach(i=1:4) %dopar% Sys.getpid()

Sear

You don't really need doSNOW for a Snow cluster; the parallel package (which comes with R) and doParallel will also handle Snow clusters. – Laniferous

@HongOoi doSNOW has certain advantages over doParallel because of restrictions in the parallel package. For example, doSNOW supports progress bars and calling the combine function on-the-fly. – Astri

One comment, I need to load library(snow). Otherwise it won't work. – Rozanna

For me, I had to make sure snow is installed, but no need to explicitly load it using load(snow) – Christopher