How to setup and use MPI on Windows with R

Question

I have downloaded an R script from the web that says that it "must be run in an MPI environment". Now I know literally nothing of MPI, except for that it is used for parallel computing, and it is rather hard to find a simple tutorial on how to set it up and use it.

Can someone give me a simple, step-by-step guide on what I should install to run this, how I should do this, and how I can run the script?

Thanks in advance.

Edit, here is what I tried:

• I installed Microsoft MPI
• I installed and loaded the RMpi package in R

Specific Problems:

> # number of available slots is passed via environment variable
> slots <- as.integer(Sys.getenv("NSLOTS"))

gives NA_integer_.

In case I use the following (i.e. I replace Sys.getenv("NSLOTS") with some scalar)

> slots <- as.integer(4)
> cl <- snow::makeMPIcluster(slots)

I get the error

Error in mpi.comm.spawn(slave = mpitask, slavearg = args, nslaves = count, : Other MPI error, error stack: MPI_Comm_spawn(cmd="C:/PROGRA~1/R/R-31~1.0/bin/Rscript.exe", argv=0x0000000009A12CA8, maxprocs=4, MPI_INFO_NULL, root=0, MPI_COMM_SELF, intercomm=0x000000000A2FF8B0, errors=0x00000000100E3C70) failed Function not implemented

So I suspect that I did not properly configure my MPI environment.

Intensity

You have to have MPI installed on your system. You can connect to it through an R package, including the "default" parallel package. See the vignette here. – Parrish

@RomanLuštrik Thank you :). The script uses parallel indeed, so I installed that package in R. What MPI package should I exactly install on my system? Is that Open MPI? I saw on the website that you can only download some tar.gz file, how do I install that? Are there other things than that which I need to do as well? Thanks again :)! – Intensity

Yes, OpenMPI is one of the MPI programs that you can use. You will have to compile the program on your own, or find a binary package that comes pre-compiled. – Parrish

@RomanLuštrik Thanks again. Could you give me any pointers on how I can do this? I have read that I would need to use my visual studio compiler for this? If you could give me some steps on how to do that that would be highly appreciated. Also, to be clear, I am using Windows (I don't know if this matters, but I mostly read about Open MPI in a Linux context). – Intensity

You can also try Microsoft MPI. Your question is way too broad for this place, I voted to close it. There are plenty of MPI tutorials around and all MPI implementations come with a manual and installation instructions. – Ostrowski

@VladimirF I already installed Microsoft MPI but still it doesn't work with Rmpi. I don't think it's that broad a question to ask how to get an MPI environment to work with R, is it? – Intensity

In that case you must include what you did and how the problem looks like. "doesn't work" tells absolutely nothing. Use edit of the question to supply all relevant information. – Ostrowski

@VladimirF I included additional details in the OP about my specific problems and the steps that I took. Could you please tell me whether this more informative? – Intensity

NSLOTS is an environment variable set by Sun Grid Engine to the number of slots granted. You are not running under SGE, therefore the environment variable is not present. MS MPI does not implement the MPI-2.0 process management functions, therefore the error message. You should search for a different MPI implementation - I can't help in you in that respect unfortunately as I myself don't use MPI on Windows. – Kutch

MS MPI doesn't support spawn, which is why you got the error. I believe you must execute an R script via mpiexec if you don't have spawn support. – Alcinia

Answer 1

try out this lines,...

> require(parallel) 
> nCores <- 8  # manually for non-cluster machines
> cl <- makeCluster(nCores) # by default this uses the PSOCK mechanism as in the SNOW package

Dramshop