How to implement a competing consumer solution?

Question

As a exercise I'm trying to find an example which implements competing consumer.

many producers - > MSMQueue <- competing consumers

So far I did not find any documentation on how to achieve this. My first attempt to figure out how is implemented in MassTransit or NServiceBus failed to many layers of indirection.

Any help will be greatly appreciated.

Capitoline

Answer 1

With MassTransit and MSMQ you can achieve this using the Distributor component.

Note that if you use MassTransit with RabbitMQ instead of MSMQ, you can implement a competing consumer scenario without using the Distributor, simply by setting the same queue name for all consumers. If you can choose between MSMQ and RabbitMQ, I'd go for RabbitMQ, as you get better management tools and SSL encryption, and it plays nicely with firewalls too.

There is some discussion on this over on the MassTransit Google Groups forum.

Foliose

As the guy who wrote the distributor, I second using RabbitMQ instead. We pump 10s of millions of messages a day through the distributor but RabbitMQ makes this so much easier. You cannot use the same queue to read from with MSMQ, each consumer needs it's own queue but the distributor component takes care of distributing a workload. – Afterclap

Thanks, I'll have a look at RabbitMQ. Travis what are the mechanics of the Distributor ? Acts as 1 single consumer in the input queue and dispatches work by workers availability using control queue ? – Capitoline

@Capitoline - correct, that's essentially how the distributor works. – Organist