Combining pub/sub with req/rep in zeromq

Question

How can a client both subscribe and listen to replies with zeromq? That is, on the client side I'd like to run a loop which only receives messages and selectively sends requests, and on the server side I'd like to publish most of the time, but to sometimes receive requests as well. It looks like I'll have to have two different sockets - one for each mode of communication. Is it possible to avoid that and on the server side receive "request notifications" from the socket on a zeromq callback thread while pushing messages to the socket in my own thread?

Remarque

Answer 1

I am awfully new to ZeroMQ, so I'm not sure if what you want is considered best-practice or not. However, a solution using multiple sockets is pretty simple using zmq_poll.

The basic idea would be to have both client and server:

• open a socket for pub/sub
• open a socket for req/rep
• multiplex sends and receives between the two sockets in a loop using zmq_poll in an infinite loop
• process req/rep and pub/sub events within the loop as they occur

Using zmq_poll in this manner with multiple sockets is nice because it avoids threads altogether. The 0MQ guide has a good example here. Note that in that example, they use a timeout of -1 in zmq_poll, which causes it to block until at least one event occurs on any of the multiplexed sockets, but it's pretty common to use a timeout of x milliseconds or something if your loop needs to do some other work as well.

Arcuate

Answer 2

You can use 2 threads to handle the different sockets. The challenge is that if you need to share data between threads, you need to synchronize it in a safe way.

The alternative is to use the ZeroMQ Poller to select the sockets that have new data on them. The process would then use a single loop in the way bjlaub explained.

Gallice

I tried threads in Ruby. The first socket.recv blocks all threads including the main (application) one... – Flywheel

@MickTaiwan Either there is an issue with the Ruby bindings, Ruby's Threads, or your code. I use threads all the time to use different sockets in Java. – Gallice

I read some examples that work with threads, but with "ffi-rzmq", and not the ruby gem "zmq". – Flywheel

It's a multi-core, multi-threaded world, folks. Get used to creating threads to handle message queues asynchronously. I've always considered it amusing that programmers think their own in-built schedulers will somehow outperform the system thread scheduler, which has been developed, used, and tested by millions of users over (in some cases) decades of use. – Rhinehart

Shut up and read the ZMQ guide - Use zmq messages to synchronize your threads! – Convert

Answer 3

This could be accomplished using a variation/subset of the Majordomo Protocol. Here's the idea:

Your server will be a router socket, and your clients will be dealer sockets. Upon connecting to the server, the client needs to send some kind of subscription or "hello" message (of your design). The server receives that packet, but (being a router socket) also receives the ID of that client. When the server needs to send something to that client (through your design), it sends it to that ID. The client can send and receive at will, since it is a dealer socket.

Wasson