MSMQ for persistence?

Question

In my project I receive a transaction from the client then process it and return a status back to the client via WCF. As you know I have to somehow save the transaction for recovery purposes and persistence.

I am thinking of using MSMQ for that purpose. When the transaction starts I will "save" it on MSMQ (database save will occur in a batch for performance reasons).

Is MSMQ good for that? Do you know of better way to create persistence? What is the best way to "backup" the transaction and keep high performance?

Kinson

Answer 1

When picking a technology I think it's helpful to consider not only can the technology meet your needs, but also whether it was designed to meet your needs. By this I mean that you should choose the best option rather than just the first option that seems good enough. You could probably solve this problem with logging or text files or various other means, but that doesn't mean you should.

My order of preference in this situation would be

1. database
2. MSMQ
3. everything else

If it isn't possible to save transactions to database for whatever reason then MSMQ can probably help you here. It should perform better than a opening a database connection & committing yet provides a 'good' persistence layer. The downside is that it's more code and another point of failure for your application (not that it will fail if written properly, but more code means more places for bugs).

You can throw your transactions into a queue very easily using something like this

private string queuePath = @".\Private$\myQueue";
MessageQueue queue = new MessageQueue(queuePath);

Message message = new Messge();
message.Id = "messageId";
message.Body = "my content";

queue.Send(message, transaction);
transaction.Complete();
queue.Close();

and then retrieve them later through querying properties: MSMQ querying for a specific message. There's a lot of other functionality out of the box but keep it simple.

Some relevent questions:

• MSMQ as buffer for SQL Server Inserts
• MSMQ vs Temporary Table Dump
• MSMQ v Database Table

Eisenach

Answer 2

I think you are looking for SQL Server Service Broker. Everything that 10 years ago we did with MSMQ we now are doing with the Service broker. It works well.

Lyonnesse

Service Broker is great but relies on database availability & response time, which may be something OP is trying to work around. – Eisenach

That applies to any transactional queuing system so it's not relevant to a specific product. – Breadthways

@ChrisSmith Was that directed at me? Use mentioning so I get notified :) In this case MSMQ would run on the webserver itself so there's no 'availability' concern - if the webserver is online, so is the message store, whereas database typically runs on another machine. And response time is great because MSMQ runs locally, while database might be somewhere halfway around the world. – Eisenach

@KirkBroadhurst: sorry missed that out :) So what happens when you need two web servers? Communication is inevitable if you need to scale. – Breadthways

@ChrisSmith That's true - we don't really know the full picture. It might be sufficient for each webserver to have it's own queue - as long as the transaction is retained 'somewhere' it may suffice. As I said in my answer, I'd always prefer to use the database as first port of call whenever possible. – Eisenach

@KirkBroadhurst: Indeed I agree. I think NServiceBus kind of bridges the gap here but I won't go further into that as it's 00:08 here and I really should be in bed :) – Breadthways

How can you interoperate between .net code and SQL Service Broker? – Slaw

Could you please answer #9702879 ? – Backhouse

Answer 3

A popular approach used by database and file systems is called Write-Ahead Logging. This is not however trivial to implement. You can find details here...

Wikipedia: Write-Ahead Logging

Hydrate

Could you please answer #9702879 ? – Backhouse