Performance comparison between ZeroMQ, RabbitMQ and Apache Qpid

Question

I need a high performance message bus for my application so I am evaluating performance of ZeroMQ, RabbitMQ and Apache Qpid. To measure the performance, I am running a test program that publishes say 10,000 messages using one of the message queue implementations and running another process in the same machine to consume these 10,000 messages. Then I record time difference between the first message published and the last message received.

Following are the settings I used for the comparison.

1. RabbitMQ: I used a "fanout" type exchange and a queue with default configuration. I used the RabbitMQ C client library.
2. ZeroMQ: My publisher publises to tcp://localhost:port1 with ZMQ_PUSH socket, My broker listens on tcp://localhost:port1 and resends the message to tcp://localhost:port2 and my consumer listens on tcp://localhost:port2 using ZMQ_PULL socket. I am using a broker instead of peer to to peer communication in ZeroMQ to to make the performance comparison fair to other message queue implementation that uses brokers.
3. Qpid C++ message broker: I used a "fanout" type exchange and a queue with default configuration. I used the Qpid C++ client library.

Following is the performance result:

1. RabbitMQ: it takes about 1 second to receive 10,000 messages.
2. ZeroMQ: It takes about 15 milli seconds to receive 10,000 messages.
3. Qpid: It takes about 4 seconds to receive 10,000 messages.

Questions:

1. Have anyone run similar performance comparison between the message queues? Then I like to compare my results with yours.
2. Is there any way I could tune RabbitMQ or Qpid to make it performance better?

Note:

The tests were done on a virtual machine with two allocated processor. The result may vary for different hardware, however I am mainly interested in relative performance of the MQ products.

Schlep

I have run simple tests months ago, with similar results. And I noticed the system is quite idle when working with RabbitMQ or Qpid. I think something must be wrong. – Deservedly

"RabbitMQ: it takes about 12 seconds to receive 10,000 messages." -- In our own tests we regularly see 20-25,000/sec ingress per CPU. So, you are doing something wrong, or using a slow client. Have you tried emailing rabbitmq-discuss with questions? – Err

Here's a good comparison, dated April 10, 2013: x-aeon.com/wp/2013/04/10/… – Bish

A comparison of performance of RabbitMQ, Kafka, HornetQ, SQS and Mongo-based replicated queues: warski.org/blog/2014/07/… – Moule

Give a try to nanomsg -- another offspring, evolved by Martin Sustrik, the ZeroMQ co-father. A good place to start to read is >>> nanomsg.org/documentation-zeromq.html – Beadroll

An updated version of RabbitMQ, Kafka, HornetQ, ActiveMQ, SQS and Mongo performance comparison is now here: softwaremill.com/mqperf – Moule

each message was how many bytes when you did this test? – Sweatt

Answer 1

RabbitMQ is probably doing persistence on those messages. I think you need to set the message priority or another option in messages to not do persistence. Performance will improve 10x then. You should expect at least 100K messages/second through an AMQP broker. In OpenAMQ we got performance up to 300K messages/second.

AMQP was designed for speed (e.g. it does not unpack messages in order to route them) but ZeroMQ is simply better designed in key ways. E.g. it removes a hop by connecting nodes without a broker; it does better asynchronous I/O than any of the AMQP client stacks; it does more aggressive message batching. Perhaps 60% of the time spent building ZeroMQ went into performance tuning. It was very hard work. It's not faster by accident.

One thing I'd like to do, but am too busy, is to recreate an AMQP-like broker on top of ZeroMQ. There is a first layer here: http://rfc.zeromq.org/spec:15. The whole stack would work somewhat like RestMS, with transport and semantics separated into two layers. It would provide much the same functionality as AMQP/0.9.1 (and be semantically interoperable) but significantly faster.

Mycenae

we will continue to use rabbitmq till someone come up with a better solution then @pieter btw. it reminds me great patents story[1]. [1] youtube.com/watch?v=5QqbDyZ8Eu4 – Jumbled

RIP mate, you built some great stuff – Vagabond

Answer 2

Hmm, of course ZeroMQ will be faster, it is designed to be and does not have a lot of the broker based functionality that the other two provide. The ZeroMQ site has a wonderful comparison of broker vs brokerless messaging and drawbacks & advantages of both.

RabbitMQ Blog:

RabbitMQ and 0MQ are focusing on different aspects of messaging. 0MQ puts much more focus on how the messages are transferred over the wire. RabbitMQ, on the other hand, focuses on how messages are stored, filtered and monitored.

(I also like the above RabbitMQ post above as it also talks about using ZeroMQ with RabbitMQ)

So, what I'm trying to say is that you should decide on the tech that best fits your requirements. If the only requirement is speed, ZeroMQ. But if you need other aspects such as persistence of messages, filtering, monitoring, failover, etc well, then that's when u need to start considering RabbitMQ & Qpid.

Despatch

ZeroMQ has no broker. You design the broker as a part of the overall design of the application and your broker listens on a zeromq, routing messages as appropriate for their destination. ZeroMQ does only one job and does it very well: message queuing. There is Malamute, which is a broker implementation designed for ZeroMQ by the ZeroMQ people, but it's not part of ZeroMQ out of the box. It's a service you can install along with ZeroMQ in it's own process or on a separate box(es) devoted to message brokering. It is it's own project. github.com/zeromq/malamute – Felid

Yes, I stated that it has no broker and linked to an article in broker vs brokerless. Was that not clear? Also, when I posted this answer in 2011 there was no Malamute, which appeared Oct 2014 – Despatch

Answer 3

I am using a broker instead of peer to to peer communication in ZeroMQ to to make the performance comparison fair to other message queue implementation that uses brokers.

Not sure why you want to do that -- if the only thing you care about is performance, there is no need to make the playing field level. If you don't care about persistence, filtering, etc. then why pay the price?

I'm also very leery of running benchmarks on VM's -- there are a lot of extra layers that can affect the results in ways that are not obvious. (Unless you're planning to run the real system on VM's, of course, in which case that is a very valid method).

Colloquialism

Answer 4

I've tested c++/qpid

I sent 50000 messages per second between two diferent machines for a long time with no queuing.

I didn't use a fanout, just a simple exchange (non persistent messages)

Are you using persistent messages? Are you parsing the messages?

I suppose not, since 0MQ doesn't have message structs.

If the broker is mainly idle, you probably haven't configured the prefetch on sender and receptor. This is very important to send many messages.

Parmentier

I am using non durable queue, I don't parse the message, Actually I am using the same code to generate messages for all three queue experiments. Changing exchange type to direct didn't have effect on performance. Also after using sender flow control (api sender.SetCapaclity(8)), the the timing got worse. without the sender flow control, the queue seems to grow unbounded. When measuring time, have you waited until receiving all the messages and the queue is fully drained? – Schlep

I found that qpid-perftest program is using Qpid's "old messaging Apis". Switching to the old Apis improved performance 10 times in my test. – Schlep

Answer 5

We have compared RabbitMQ with our SocketPro (http://www.udaparts.com/) persistent message queue at the site http://www.udaparts.com/document/articles/fastsocketpro.htm with all source codes. Here are results we obtained for RabbitMQ:

Same machine enqueue and dequeue:

"Hello world" --
Enqueue: 30000 messages per second;
Dequeue: 7000 messages per second.

Text with 1024 bytes --
Enqueue: 11000 messages per second;
Dequeue: 7000 messages per second.

Text with 10 * 1024 bytes --
Enqueue: 4000 messages per second;
Dequeue: 4000 messages per second.

Cross-machine enqueue and dequeue with network bandwidth 100 mbps:

"Hello world" --
Enqueue: 28000 messages per second;
Dequeue: 1900 messages per second.

Text with 1024 bytes --
Enqueue: 8000 messages per second;
Dequeue: 1000 messages per second.

Text with 10 * 1024 bytes --
Enqueue: 800 messages per second;
Dequeue: 700 messages per second.

Madame

Answer 6

Try to configure prefetch on sender and receptor with a value like 100. Prefetching just sender is not enough

League

Fro qpid, I had the impression that Receiver.setCapacity(100) sets the prefetch for receiver. Afer doing that the performance improved 10 times for the code using "new qpid api" and the performance bacame similar to the Qpid old messaging Api. I have updated the post with the result. However Qpid still seems to be 4 times slower than rabbitmq. – Schlep

Answer 7

We've developed an open source message bus built on top of ZeroMQ - we initially did this to replace Qpid. It's brokerless so it's not a totally fair comparison but it provides the same functionality as brokered solutions.

Our headline performance figure is 140K msgs per second between two machines but you can see more detail here: https://github.com/Abc-Arbitrage/Zebus/wiki/Performance

Sorcim