How can we speed up receiving messages from MSMQ?

Question

My application's bottleneck has become sending and receiving messages over MSMQ with MassTransit. The send and receive are both happening within the same application, but there are often too many messages to use an in-memory queue.

Here is my simple queue setup:

messageBus = ServiceBusFactory.New(sbc =>
                {
                    sbc.UseMsmq();
                    sbc.VerifyMsmqConfiguration();
                    sbc.UseMulticastSubscriptionClient();
                    sbc.ReceiveFrom("msmq://localhost/msg_queue");
                    sbc.Subscribe(subs =>
                    {
                        subs.Handler<EventMessage>(msg => Enqueue(msg));
                    });
                });

For my experiments, MSMQ currently has ~1 million messages and we are not pushing any new messages to it. We are not doing any work in Enqueue() except time how quickly messages are being sent.

With that information, we can only grab between 150 and 200 messages per second from MSMQ, when I'd hoped it would be at least 1000 messages per second when there is no network latency. Each messages is <2kb.

How can we speed up how quickly MSMQ passes messages to an application over MassTransit while maintaining the message ordering enforced by a queue?

Rufescent

You may have already been through this, but have a look here. We use Websphere MQ and probably get about what you're getting right now with MSMQ. – Tintype

What is your 1,000 messages/second hope based on? – Tirzah

If you currently have a backlog of a million unprocessed messages then don't expect miracles, the datastore for them is substantial. MSMQ has no trouble handling thousands of messages per second. This went bad a while ago, best to purge the queue to get back to a reasonable starting point. – Pillsbury

@DStanley It's based on some blog posts I read in my research that claimed thousands of messages per second. It's not crazy to expect to be able to pass 1000 small messages back and forth through a queue in a second, especially with no network latency. – Rufescent

To maintain the order of the messages I understand it so that you are reading them synchronously, right? could the bottleneck be in the processing of the messages, not in reading out of the queue itself? – Frances

@JensH I don't think so. I set up a test application that just reads them off and times how quickly we're receiving messages, so there is no other heavy processing going on. I say in order because I considered receiving on multiple threads and pushing to an in-memory queue, but that kind of processing can happen in any given order. – Rufescent

Answer 1

I did addressed something similar before. If I remember it correctly, We specified message expiration through TimeToBeReceived (The total time for a sent message to be received from the destination queue. The default is InfiniteTimeout.) . Refer this msdn link.

Sacramental

Isn't this just expiring messages if your processing isn't fast enough? If you can only process 200 messages per second, I don't think deleting 800 more messages qualifies as processing 1000 per second. Am I missing something with this configuration? – Rufescent

@Chris, I am with you. However, in my system (the disconnected system), we ignore messages or calledbackogs based on two strategy. (a)TimeToBeReceived (b) a custom timestamp along with message. Using these factors, we were able to process latest message with less backlogs in queue. Please keep in mind, processing 500-1000 messages per second is achievable provided you have a better mechanism to clean the backlog. – Sacramental