How should I set rebus up for one producer and many consumers

Question

I am going through the samples and reading the docs but I am still not sure of how to configure rebus for my scenario (or that using the bus is a good fit).

I have one producer of Tasks to do, lets say ImportOrder and CalculateOrderPrice

I want to dump messages from the producer and queue lots of these messages. I want two clients that listens to ImportOrder, and 10 clients that listens to CalculatePriceOfOrder. I do not want the same order to go to multiple endpoints at the same time, I am trying to spread the workload.

Producer config so far:

 <rebus inputQueue="publisher.input" errorQueue="publisher.error" workers="1" maxRetries="5">

var adapter = new BuiltinContainerAdapter();
            Configure.With(adapter)
                   .Logging(l => l.Log4Net())
                        .Transport(t => t.UseMsmqAndGetInputQueueNameFromAppConfig())
                        .Subscriptions(s => s.StoreInXmlFile(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "rebus_subscriptions.xml")))
                        .MessageOwnership(d => d.FromRebusConfigurationSection())
                        .CreateBus()
                        .Start();

Consumer config so far:

<rebus inputQueue="calcserver1.input" errorQueue="calcserver1.error" workers="1" maxRetries="5">
<endpoints>
  <add messages="Messages.RecalculateContractMessage,  Messages" endpoint="[email protected]"/>
</endpoints>

Configure.With(adapter)
               .Logging(l => l.Log4Net())
                .Transport(t => t.UseMsmqAndGetInputQueueNameFromAppConfig())
                .MessageOwnership(d => d.FromRebusConfigurationSection())
                .CreateBus()
                .Start();

        adapter.Bus.Subscribe<RecalculateContractMessage>();

I cant seem to configure this setup, it does not really matter if I use msmq or sqlserver.

• Does rebus (or any servicebus type of solution) sound like a good fit here?
  
• Should I use pub/sub or plain Send pattern? And will that choice affect if messages gets processed by only one endpoint at the time or several?
  
• Could anyone point me to a good example or explain how to set this scenario up?

Guernsey

Answer 1

I'll answer your questions one by one:

• Does rebus (or any servicebus type of solution) sound like a good fit here?

Yes, a service bus can help you deal with lots of stuff (e.g. serialization, threading, etc.) that you would otherwise have to somehow handle on your own.

• Should I use pub/sub or plain Send pattern? And will that choice affect if messages gets processed by only one endpoint at the time or several?

In this case, it sounds to me like you're in need of a competing consumers type of distribution of your tasks.

This is NOT pub/sub - pub/sub implies 0..* consumers for each message instance, but you want exactly one recipient for each task. Therefore, you would bus.Send the tasks in this case.

• Could anyone point me to a good example or explain how to set this scenario up?

I've written several blog posts in an attempt to show how you can scale out the handling of your workload with Rebus with examples for the different supported transports.

It should be fairly easy for you to get started by looking at the posts and use e.g. SQL Server as your transport.

Ferne

Thanks a lot, I read the series and they are great :) I'll start with the sql server scaleout solution, it sound like a good fit – Guernsey

A follow up question: Say that we have two different messages and 0..* workers for each message. Would two different queues be the solution? Attempting to handle different messages on same bus seems to create lots of issues. – Guernsey

I think I got it right now, Using bus.Advanced.Routing.Send to specific queues and the setting up each worker type to consume from that queue. If this is the proposed way to go then perhaps I could contribute with a sample if you want :) – Guernsey

What do you mean with "Attempting to handle different messages on same bus seems to create lots of issues"? – Ferne

I think I had a bit of a strange setup, two different messages in the same asseembly, lets say OrderProductMesage and RegisterMemberMessage. I Dumped messages of both types in the same queue. I then created two separate workers that consumed messages from that queue. That resulted in routing errors. But when I started using a queue per message and setting int up a bit more explicit it works very well :) – Guernsey

Ah, right! When workers are processing messages off of the same queue, the workers must be instances of the same worker process - but you seem to have figured that out allright ;) – Ferne