Does RabbitMq do round-robin from the exchange to the queues

Question

I am currently evaluating message queue systems and RabbitMq seems like a good candidate, so I'm digging a little more into it.

To give a little context I'm looking to have something like one exchange load balancing the message publishing to multiple queues. I don't want to replicate the messages, so a fanout exchange is not an option.

Also the reason I'm thinking of having multiple queues vs one queue handling the round-robin w/ the consumers, is that I don't want our single point of failure to be at the queue level.

Sounds like I could add some logic on the publisher side to simulate that behavior by editing the routing key and having the appropriate bindings in place. But that's kind of a passive approach that wouldn't take the pace of the message consumption on each queue into account, potentially leading to fill up one queue if the consumer applications for that queue are dead.

I was looking for a more pro-active way from the exchange entity side, that would decide where to send the next message based on each queue size or something of that nature.

I read about Alice and the available RESTful APIs but that seems kind of a heavy duty solution to implement fast routing decisions.

Anyone knows if round-robin between the exchange the queues is feasible w/ RabbitMQ then? Thanks.

Irkutsk

As soon as we tested a single queue in production we slammed into a scale problem. A single queue is single threaded. It can't keep up with our load. We're looking for best practices to work in a round robin distribution – Thoroughwort

Answer 1

Exchanges are generally stateless in the AMQP model, though there have been some recent experiments in stateful exchanges now that there's both a system for managing RabbitMQ plugins and for providing new experimental exchange types.

There's nothing that does quite what you want, I don't think, though I'm not completely sure I understand the requirement. Aside from the single-point-of-failure point, would having a single queue with workers reading from it solve your problem? If so, then your problem reduces to configuring RabbitMQ in an HA configuration that permits you to use that solution. There are a couple of approaches to doing that: either use HALinux and a shared store to get active/passive HA with quick failover, or set up more than one parallel broker and deduplicate on the client, perhaps using redis or similar to do so.

I suggest asking your question again on the rabbitmq-discuss mailing list, where more people will be able to offer suggestions, and where the discussion can be archived for posterity.

Bunche

Thanks for the help. That's a good work around for now as I'm only at the evaluating stage. – Irkutsk

Answer 2

One built in way you can do a form of sharing a form exchange to queues, but not exactly round robin, is Consistent Hashing. rabbitmq_consistent_hash_exchange

How too https://medium.com/@eranda/rabbitmq-x-consistent-hashing-with-wso2-esb-27479b8d1d21

Paper to explain, it puts queues at a weighted distribution on a circle and then by sending random routing key it will send to the closest queue. http://www8.org/w8-papers/2a-webserver/caching/paper2.html

Fadden

Answer 3

Agree with Tony on the approach.

Here is a 'mashup' of RabbitMQ, Redis that you could use instead of rolling your own - http://xing.github.com/beetle/

Filing