zmq vs redis for pub-sub pattern

Question

redis supports pub-sub
zmq also supports pub-sub via a message broker

What would be the architectural pros\cons for choosing between them?
I'm aiming at points which are beyond the obvious use-case specific performance benchmarking that should be done (here's a nice example).

Assume use of a high-level language such as Python.

Georgettageorgette

Hey, it would be great if all StackOverflow answers were tailor made and in depth like the nice example you referenced :) But the way it is makes your question too open ended. Perhaps you should try and nail down some specific differences or trade-offs that you're agonizing about and then re-post with more surgically targeted questions. – Barahona

But thanks for the blog link....it looks like a great article. – Barahona

Is this for n-to-n pub/sub? – Yasmineyasu

@Yasmineyasu - yes n-to-m use case. btw, are there use cases besides n-to-m you would use pub-sub for? – Georgettageorgette

Is ZeroMQ LGPL license something you need to consider ? wiki.zeromq.org/area:licensing – Chaschase

Here is a good comparison including some benchmarks: gist.github.com/hmartiro/85b89858d2c12ae1a0f9 – Dalhousie

Answer 1

I have worked with both ZeroMQ and Redis with python. I would say ZeroMQ is more robust, it offers real simple load balancing and also more than pub-sub, like request reply among others. But if you are only after pub-sub, redis is much simpler.

In case the redis server crashes or stops working, all the clients will stop working as well, with ZeroMQ, the clients work even if there is no server.

Both services are available with any programming language, ruby, python, C, C++ and more.

In short, redis is much simpler, very reliable. ZeroMQ is extremely reliable but more complex.

If I was only doing pub sub, I would pick redis, else I would pick ZeroMQ. If I would forsee huge loads of traffic, I would pick ZeroMQ

Erlene

How does ZeroMQ handle server crashes then? – Asarum

Not sure how they do it. But you can start a client, and post messages, even if there is no server running. You start the server and things still happen. All I know is that it works great and is very resilient and robust and handles crashes very well, though within reason – Erlene

IIRC, It runs a separate thread/process that keeps messages in that thread. So you're never really directly interfacing at the socket level. The messages and disconnects are all conveniently handled in the ZMQ thread, asynchronously, while your application does it's work. – Colloidal

Answer 2

ZeroMq Pros/Cons

• Pub/sub peers can connect and disconnect independently; messages are saved to buffers based on HWM settings, automatically sent upon peer availability (store-and-forward)
• If a peer fails, buffered messages will be lost
• Topic subscriptions support prefix matching with pub/sub enveloping only; NEWS subscription matches NEWS* messages

Redis Pros/Cons

• AOF snapshotting to disk persists messages in the event redis fails
• Pub/sub clients depend on redis for connectivity
• Wildcard matching for selective topic subscriptions like news.* supported

Yasmineyasu

Answer 3

I have been researching this myself as I needed to decide whether to use Redis pubsub or ZMQ pubsub for communication layer for distributed systems. I think Redis and ZMQ differ in terms of how application is setup.

1. ZMQ pubsub innately connect directly i.e. no middle man.
   You can create middle-man like instance such as forwarder device

2. For Redis pubsub, both subscriber and publisher need to connect to Redis.

The lack of middle-man in ZMQ means subscriber needs to somehow know to connect to publisher to get the message. In my system where application spawn publishers that needs to send info to subscribers, there is no way of doing that without having forwarder device that subscribers connect to before my application starts.

Latency matters in my system as I want remote boxes to do things as fast as it can.

Zmq (direct pubsub)
avg: 0.000235867897669
max: 0.0337719917297
min: 0.000141143798828

Zmq (w/ forwarder)
Avg: 0.00237249334653
max: 0.00536799430847
min: 0.000249862670898

Redis (8gb ram)
avg: 0.000687216520309
max: 0.0483138561249
min: 0.000313997268677

Redis (32gb ram)
avg: 0.000272458394368
max: 0.00277805328369
min: 0.000216960906982

• if your application is on the subscriber side where there is publisher daemon that you want to get info from, then i would go for ZMQ because you can directly connect to publisher.
• if your application is on publisher side, then i would go to Redis pubsub because subscribers are already connected to Redis listening.

Pagas

Do you happen to recall units for those numbers? seconds? milliseconds? – Catinacation

Answer 4

Here's how I would decide. Make minimal test cases using each product. See which is easier to build and works better. Push each test case a little further and then discard one line as too much work.

Bushnell

I actually did, but I rather consult the crowd for things I may find out only down the road... initial hurdles could be misleading – Georgettageorgette