Why do we need service bus frameworks like NService Bus/MassTransit on top of message queuing systems like MSMQ/RabbitMQ etc?

Question

In the distributed message transaction world, am trying to understand the different parts that are involved in developing distributed systems. From what I understand you can design messaging system using enterprise bus backed with a message queue system. Why is it a good idea to use both? Can the same be achieved by programming against just the message queuing system? What are the advantages of using both together?

Felty

Answer 1

You certainly can code directly against the messaging infrastructure and you will find that there are pros and cons w.r.t. each transport. There are many decisions that you will need to make along the way, though, and this is where a service bus may assist.

Developing directly against the queuing system will inevitably lead to various abstractions that you will require to prevent duplication.

A service bus will provide opinions/implementations for:

• Message delivery
  • exactly-once (distributed transactions - distributed transactions are not supported by all queuing systems)
  • at-least-once (non-transactional)
  • at-most-once (will probably require some transactional processing but you can get away with no distributed transactions)
• Retrying failed messages
• Request / Response
• Message distribution
• Publish/Subscribe (probably quite easy with RabbitMQ directly, not so much with MSMQ directly)
• Message Idempotence
• Dependency Injection

Some service bus implementations provide a framework for implementing process managers (called sagas by most). My current opinion is that a process manager needs to be a first-class citizen as any other entity is but that may change :)

Anyhow, if you as still evaluating options you could also take a look at my FOSS project: http://shuttle.github.io/shuttle-esb/

So a service bus may buy you quite a bit out-of-the-box whereas coding against the queues directly may be a bit of work to get going.

Cittern

"a service bus may buy you quite a bit out-of=the-box whereas coding against the queues directly may be a bit of work to get going". Great summarization. Thanks! – Felty

Publish/Subscribe is easy on MSMQ using MassTransit. Well, as easy as anything on MSMQ is, anyway.. – Caundra

@stuartd: I meant that when using RabbitMQ directly pub/sub is quite easy whereas with MSMQ directly it is not. Using a service bus like shuttle-esb certainly makes life easier :) – Cittern

@EbenRoux yup, ServiceBus FTW. Your meaning wasn't entirely clear in your answer. – Caundra

@stuartd: I edited the answer to make that more clear, thanks. – Cittern

Answer 2

I can't comment directly on MassTransit, having only tinkered with it.

I use NServiceBus and am a fan of it. I think there are valid reasons for directly using queuing technology, but I think rolling your own ESB using MSMQ/RabbitMQ would cost a lot more than simply using a commercial product (or open source product e.g. MassTransit).

So do you need it? No. Will it make your life much easier if the features match your requirements? Absolutely.

Roughage