Why is MSMQ faster than WCF QueueService?

Question

I have self-hosted WCF service in console with netNamedPipeBinding binding. The service has just one empty method Send(DataTable bulk)

[ServiceContract]
public interface IWcfQueueService
{
    [OperationContract]
    void Send(DataTable bulk);       
}
public class WcfQueueService : IWcfQueueService
{
    public void Send(DataTable bulk)
    {           
       // Here would be something like _bulks.Add(bulk);
       // BUT, for now it is empty method and still it's slower than MSMQ
    }    
}

My client gets 200K inputs from DB and process it with our BoundedThreadPool(only creates, let say, 20 threads). Each input is processed with different thread. Each thread executes MyMethod and in the end of MyMethod the result is added to bulkManager.

public void MyMethod(string input)
{            
    var res = ProcessInput(input);
    bulkManager.Add(res);
}

When bulkManager accumulates N items (=bulk) it pass the bulk to another thread that all it does is enqueue that bulk with one of two methods:

1. If wcf enabled: wcfQueueService.Send(bulk);
2. else if MSMQ enabled: new MessageQueue(@".\private$\q").Send(new Message {Body = bulk});

All two methods works, but MSMQ works much more faster. With MSMQ client manages to process about 80K bulks in 20seconds while with wcf only 20K-30K bulks. I don't understand why it happens. My WCF runs in different process like MSMQ does. In addition, my WCF doesn't stores anything, it has empty method. So why MSMQ wins WCF?

Updated

As leppie suggested I tried .NetRemoting. NetRemoting indeed improved the speed. The client processed 60K. But,

1. It's still slower than MSMQ
2. As I read .Net Remoting is deprecated by WCF and WCF should be faster than .Net Remoting according to this, so why I get that my wcf is slower? Maybe my binding is wrong?

Bergman

You will probably find remoting is a lot faster than MSMQ (given named pipes, I assume IPC on the local machine). – Mayan

@leppie, .Net Remoting indeed improved the speed. The client processed 60K. But, 1 - it's still slower than MSMQ and 2- as I read .Net Remoting is deprecated by WCF and WCF should be faster than .Net Remoting according to msdn.microsoft.com/en-us/library/…, so why I get that my wcf is slower? – Bergman

@theateist: Can you change your InstanceContextMode on the service to Single and ConcurrencyMode to Multiple? Then do another test. Also try to adjust the throttling settings (in the service behavior). – Selfknowledge

@theateist: WCF only 'replaces' .NET remoting in non-IPC scenarios. WCF is disconnect by nature while remoting is 'mostly connected'. .NET remoting will NEVER die! :) – Mayan

@leppie: Come on, man... What are you talking about? It's already dead :). – Selfknowledge

Please post the ServiceBehavior – Ponderable

Have you tried using the netTcpBinding? If that's also slow you can likely eliminate the bindingtype as the problemcause. – Oscillograph

are you using msmqBinding in your WCF? – Doviedow

A difference could be the synchronous behavior of WCF. If you call a void function over WCF, the calling side will block until the receiving end has fully processed the message, so that exceptions can be marshalled back. This can be extra work for both ends. Edit: this can be turned off: google for OperationContractAttribute.IsOneWay. – Picot

Answer 1

You are not comparing like with like.

The most obvious difference is that in the WCF case your timings include execution of the entire service-side channel stack and operation invocation, whereas your direct MSMQ case only measures the time to enqueue the payload on the client side. Bear in mind that the service side processing in WCF includes deserialization of your DataTable object, which is probably quite expensive if your bulking factor N is large.

What is more, depending on how you have configured your service instancing and throttling knobs, the requests from the client may be being made more quickly than the service is configured to handle, resulting in the requests themselves being queued for execution on the service side.

Also, depending on how you have configured your binding, there may be other significant differences, security for example. By the way, did you really use NetNamedPipeBinding (as your question states) rather than NetMsmqBinding as the title seemed to imply? If so, using the default binding configuration you will have totally unnecessary encryption and signing of each Bulk message going on, which isn't happening in your direct MSMQ case. These cryptographic operations on large messages will be relatively expensive also.

A better comparison would be with the WCF operation defined as OneWay.

Vanir

that 'totally unnecessary' link is dead, any chance you have a copy of the entry? really interested to read that, thanks – Conveyance

Answer 2

Can you provide example code that show the behavior you are seeing?

I did my own testing by generating 20,000 messages to send. I tried both, 20,000 with direct MSMQ, and 20,000 with WCF abstracting the MSMQ endpoint for me.

The 20,000 with direct MSMQ used 64.75 percent of the CPU time and the WCF version sending 20,000 messages used 34.16 percent of the CPU time (instrumented using the Analyze feature of Visual Studio Ultimate).

Unless I have made an error on my end the WCF version was almost twice as fast as the hard coded MSMQ equivalent.

Consanguinity

Answer 3

I think the difference is what happens in your WCF Operation vs. what MSMQ does when it accepts a request.

I'd expect that the method that is executed when you enqueue a message with MSMQ simple accepts the message and nothing else. Some back-end worker does the heavy lifting.

In your Operation, you need to pass the request to a singleton instance of a request handler. The request handler instance should be instantiated when the service is "spun up".

You may also find improved performance by using Task Parallel Library in .NET 4 to parallelize your requests.

Dispend