Will RStan run on a supercomputer?

Question

Stan is a new Bayesian analysis software by Gelman et al.

RStan is, I am guessing, a way to call Stan from within R.

Will Stan / RStan run on a supercomputer with a Linux operating system, and if so can it take advantage of the super-computer's multi-processors? I have been told that WinBUGS will not run on a Linux machine and/or cannot take advantage of a supercomputer's multi-processors.

I am looking for a way to speed up Bayesian analyses - from weeks to days / hours.

Hupp

What does the documentation of stan or rstan say about this? – Orpheus

the first google hit for "stan gelman" (the first is a lawyer named Stanley Gelman) is slides from a talk that says that Stan incorporates a parallel sampler that "generates openMP code, parallel C API for multi-core". i.e. it would run on a multi-core machine, I don't know if it would run under MPI or other parallel architectures ... – Blowbyblow

Thanks. These are the slides I saw before posting here. stat.columbia.edu/~gelman/presentations/stantalkhandout.pdf I guess I was looking for the word 'Linux' or 'supercomputer'. If my post is still active tomorrow or next Monday I will post here what I learn by then. – Hupp

I have not found any documentation for RStan on the Cran package website: cran.r-project.org/web/packages/available_packages_by_date.html – Hupp

I think "supercomputer" and "Linux" are actually unlikely search targets. Most clusters run Linux, so that will be taken for granted; similarly, I don't often see cluster- or cloud-computing people using the term "supercomputer". You might be looking for "parallel", "MPI", and other buzzwords found (for example) cran.r-project.org/web/views/HighPerformanceComputing.html – Blowbyblow

a little more information: JAGS runs fine on clusters, and seems to have some parallel capabilities: andrewgelman.com/2011/07/parallel-jags-rngs . WinBUGS can be made to run under WINE, on at least one cluster: wiki.hpc.ufl.edu/doc/WinBUGS -- although parallel-Stan will certainly be faster than parallel-JAGS or cluster-WinBUGS ... – Blowbyblow

@BenBolker Thanks for help, Ben. Below I have posted what I have learned so far. – Hupp

I have two doubts about this question: If I am not mistaken each chain is executed serially, its a differential equation being solved, honestly how much faster can one chain get? But maybe there are gains from faster memory and L1/L2 caches for your use case. Secondly: With regards to the author's efforts that are in an answer below; if each chain only requires additional memory (assuming CPU speed can't be improved beyond a certain limit), why not use try light weight VM's like dockers on a supercomputer instead with the stuff you want preinstalled? – Naker

Answer 1

Stan and rstan should run on Linux, Mac, or Windows that supports the dependencies. We have not tested on BSD or Oracle, but we expect them to work with either the g++ or clang compilers (although not the Oracle compilers).

There is no explicitly parallel code in Stan or rstan but neither is there any code that prevents the binary from being executed by several processes simultaneously. For example, if you use Stan from the command line in a bash shell, you could do something like

./my_model --data=my_data.dump --seed=12345 --chain_id=1 --samples=samples_1.csv &
./my_model --data=my_data.dump --seed=12345 --chain_id=2 --samples=samples_2.csv &

and so forth for as many chains as you like. It is important to use the same seed but different chain_id when executing in parallel.

If you are using the rstan package, you can call the main stan() function using any of the parallel engines supported by R and your operating system. Again, it is best to pass the same seed and different chain_id. As of rstan v1.0.3 (not released yet), there is a function called sflist2stanfit() that takes a list of stanfit objects that may have been generated in parallel and combines them into a single stanfit object for analysis.

For more information, there is a thread devoted to parallel execution at

https://groups.google.com/d/topic/stan-users/3goteHAsJGs/discussion

Melvamelvena

Not much has changed. For RStan, it is easier in May 2016 in the sense that you can specify the cores argument to stan or sampling or just set options(mc.cores = parallel::detectCores()) prior to calling stan or sampling. But that only reduces typing. It still does the same thing (parallelizes the chains) and does not parallelize anything within chains. – Melvamelvena

Answer 2

I wrote that I would post what I learned.

The university Supercomputing Center believes that RStan will run on their machines. However, I must apply for an account, which might take some time. So, I will not be certain that RStan will run on those machines for a while yet. For what it is worth the formal name of their facility is the 'Arctic Region Supercomputing Center'.

I had trouble installing RStan on my desktop and had to get OIT assistance. So, here are the steps I used and the code used by the OIT gentleman. I have a Windows 7 Professional operating system.

1. I had to use R 2.15.1

2. I installed R in the directory 'C:\R\R-2.15.1' so there would be no spaces in the directory name

3. I had to install Rtools.

4. I installed Rtools in the directory 'C:\Rtools'

5. Make sure that Rtools appears in the path so that R can locate the C++ compiler in Rtools

6. To check:

   Computer, Properties, Advanced System Setting, Environment Variables, Path.

   I think I should include both: 'c:\Rtools\bin' and: 'c:\Rtools\gcc-4.6.3\bin'

7. Open R

8. Here is the R code to type (this code appears here: http://code.google.com/p/stan/wiki/RStanGettingStarted):

   install.packages('inline')

   install.packages('Rcpp')

   install.packages('RcppEigen')

   options(repos = c(getOption("repos"), rstan = "http://wiki.stan.googlecode.com/git/R"))

   install.packages('rstan', type = 'source')

   library(rstan)

9. Then I ran the school example from here:

http://code.google.com/p/stan/wiki/RStanGettingStarted

Last week I had been trying to install STAN using instructions contained within the pdf file 'stan-reference-1.0.2' instead of the instructions at the above link.

I hope this helps others. If and when I learn whether RStan definitely will run on the Supercomputing Center machines I will post here what I learn.

I have not uninstalled STAN to test the above procedure. Hopefully I did not make any errors in the above steps.

Hupp

for what it's worth, it's highly likely that any sensibly set up cluster (and most Unix desktops) will have all of the equivalents of your steps #2-6 done already ... – Blowbyblow

All you need is a late enough version of R and a recent enough C++ compiler. We've made RStan much easier to install since 2012 and improved the doc for various platform installations with more information on prerequisites. – Sonorous

I am curious if this super computer run docker? If you have too much trouble installing stan AND if someone has already installed docker for the kind of machines you have then it maybe easier playing with docker on your laptop, building the image and deploying it in either cluster mode or something.. – Naker

Answer 3

Here's a concrete parallelization function that takes source code as text:

library(rstan)
library(parallel)

parallel_stan <- function(code, data, cores=detectCores(), chains=8, iter=2000, seed=1234) {
    cat("parallel_stan: cores=", cores, ", chains=", chains, ", iter=", iter, ", seed=", seed, "\n", sep="")
    cat("--- Step 1: compile the model (and run it once, very briefly, ignoring its output)\n")
    f1 = stan(model_code = code, data = data, iter = 1, seed = seed, chains = 1, chain_id = 1)
    cat("--- Step 2: run more chains in parallel\n")
    sflist <- mclapply(
        1:chains
        , mc.cores = cores
        , function(i) stan(fit = f1, data = data, iter = iter, seed = seed, chains = 1, chain_id = i)
    )
    # ... passing the same seed to all chains follows example(sflist2stanfit)
    # ... important to use the same seed but different chain_id when executing in parallel
    cat("--- Finished.\n")
    return(sflist2stanfit(sflist))
}

Demagogy

Answer 4

The first hit on an RSeek search (for: Rstan gelman) yielded this after following one link:

https://github.com/stan-dev/rstan/wiki/RStan-Getting-Started

It's not yet on CRAN.

Valenti

Answer 5

This is a general comment about Bayesian MCMC calculations.

Typically supercomputers run server class processors rather than desktop class. Stan and other MCMC programs are almost always pretty strictly serial on a per-chain basis, i.e. you will rarely get any possibility of a speedup for a single chain by having more than one processor. We have a small cluster with a dual-Xeon-class server and a couple of regular desktop machines as workstations. The Core-i7s processors in the workstations are typically about 40% faster than the server for doing real-world calculations as long as you stay inside their 16GB RAM limitation.

The fastest machines for doing these sorts of calculations will likely be over-clocked custom game machines with a water-cooled CPU.

That said you can of course run different chains in parallel as has been pointed out above.

Samsara