Passing messages between remote MailboxProcessors?

Question

I'm using MailboxProcessor classes in order to keep separate agents that do their own thing. Normally agents can communicate with one another in the same process, but I want agents to talk to one another when they are on separate processes or even different machines. What kind of mechanism is best for implementing communication between them? Is there some standard solution?

Please note that I'm using Ubuntu instances to run the agents.

Sweeps

Who voted this as 'off-topic'? If you don't understand the question please don't interfere with it. – Osbert

#502156 may provide some insight. – Winnebago

maybe look at windows azure queues. Its what they use to let different machines (they call them worker roles) talk to each other. What you can do is have an agent on each machine that is in charge of communicating with the others to either request work, notify work is done etc.. – Lipinski

@Lipinski interesting, but I'm not using Azure. I've already figured out that I need to set up my own type of queuing infrastructure – Sweeps

Answer 1

I think you're going to have write your own routines to serialize messages, pass them accross the process boundaries and then dispatch them on the other side. This will also require a implementation of a ID system where each mailbox has an ID and processes can send messages to IDs instead of just Mailbox.Send. This is not easy, as local boxes will be able to access local memory, but remote mailboxes will not.

I would look at something like RPyC (http://rpyc.wikidot.com/) as it provides a protocol somewhat like you are looking for.

Basically the answer is 'no' there isn't really a good way to do this.

Gawlas

do you know how this would go in other languages which have agents: Erlang, Scala and co? Thanks! – Lipinski

Well, in Erlang, you don't have shared memory. Each "thread" is really a separate process. So when you say pid ! "foo" pid is simply a id of a process that "foo" gets sent to. Since there is no shared memory, and processes have pids already, networking it is as simple as creating a dictionary of pid to IP mappings and passing messages based on that information. The problem with this method is that all communication has to be done via message passing, and currently also involves a memory copy of the message. Performance-wise this isn't always best. I don't have any experience with Scala. – Gawlas

And I should claify, Erlang processes are not OS processes, they are VM processes. A Erlang process has a memory overhead of a few hundred bytes. A OS process has an overhead in the multi-KB range. – Gawlas