Setting up an R-Server that supports asynchronous communication with clients

Question

In short

Is it possible set up an R-Server that can handle/dispatch multiple client requests in an asynchronous way?

I guess I'm looking for some sort of socket communication. Or is there something even more efficient to let R talk to other applications?

Now, I don't really care if the communication is eventually realized via "plain-vanilla socket communication" (something like socketConnection(port=6011, server=TRUE) for the server process, socketConnection(host=Sys.info()["nodename"], port=6011) for client process(es) in combination with writeLines() and readLines() of JSON character-strings) or something more "advanced" like using web server facilities based on HTTP requests.

More details

First of: My background with respect to those nitty-gritty informatics details is all "DIY", so things like socket communication, details about master-server concepts or asynchronous communication etc. are pretty new to me. So please don't bite ;-)

I'd like one of my processes to act as an R-Server. AFAIU, R was not designed under a "multi-threading-paradigm". So I wonder how I can make my R-Server handle concurrent client requests (currently coming from a third party software that sends its requests/objects via JSON-style character strings) in an asynchronous way.

In plain english I would like to tell my server process something like this:

Once you get an incoming message at port xy, do what you need to do but immediately switch back to "responsive mode" to be able to handle the next client request and always take care that each client gets the results that is logically linked to its respective request.

Homework

I started out with something very basic:

con <- socketConnection(port=as.numeric(port), server=TRUE)

This would make my R process become a server. It works a treat, but since con is my "only" connection object, the server is doomed to handle each client request sequentially:

Read the input from the connection, do some calculations, write output back to the connection and only then handle the next client request.

I thought about have my server process immediately dispatch everything that needs to be done to an auxiliary Rscript process so that the "main" server process is ready to take the next request. But since at the end of the day the results need to pass through con again (in my "main" server process), I think that doesn't really get me any points as the server process still needs to wait until the auxiliary process is finished before taking the next request. Or can I tell this auxiliary process to "directly" report back to the client request somehow?

Even though I'm currently not aiming at turning my server into a full-grown web server, these sophisticated web server approaches seemed very promising to me at a first glance:

• RServe (CRAN page)
• Rook (CRAN page)

Is asynchronous client-communication possible with one of these?

Any pointer whatsoever will be greatly appreciated!

---

EDIT

I'll offer a bounty of 300 credits once the question is eligible.

Various

I think this question merits a bounty – Starlight

@agstudy: you're right. Consider it done (as soon as this question is eligible for bounty, I'll put 300 credits on the counter) ;-) – Various

Try please to explicit the Asynchronous need? juste e.g adding a use case (typical need) in you question? the client ask for a task and don't ...when the task is finished...he recieves an alarm... – Starlight

Alright, I'll try to come up with a nice use-case once I put the bounty up. – Various

Answer 1

Rserve was designed to do precisely that.

It supports many clients, R being one of them. In the RSclient package you can do RS.eval(.., wait=FALSE) to execute the evaluation in the background. You collect the result using RS.collect where you can specify a list of connections.

See also Rserve.cluster package for dispatching parallel processing to Rserve instances.

If you also want asynchronous communication the other way (from server to client), see OOB commands in Rserve (those are unsolicited commands executed on behalf of the server, not the client). There is now also proxy support if needed. Please ask questions on the stats-rosuda-devel mailing list.

Alginate

Fantastic! Thanks a lot for the quick answer! – Various