Patterns for implementing transactions outside of a database

Question

I have to send an email, write to a file, and call a web service. To maintain consistency, all steps must happen. If any step throws an exception or errors out, all steps must be rolled back.

Before I go rolling my own object ACID engine, are there any commonly accepted patterns for implementing ACID semantics at the object level?

Better yet, are there any existing libraries I can use for the the .NET platform?

Edit: I know sending an email can't be undone, but failing to connect to the SMTP server is cause to kill the whole transaction. Also, I'd like this to be extensible for use with future actions.

Caesarea

How do you roll back an email? – Warmedover

A retraction would work. In theory, I could pluck it out of the Exchange server as if it never happened, but that could get fishy. – Caesarea

Doubt it would work. If the e-mail has been read, it usually cannot be recalled. If it's an SMTP send, you'll (just about) never be able to recall it. – Middleweight

Either way, failing to connect to the SMTP relay is cause enough to kill the transaction, and that can be detected. – Caesarea

The point is, you need a defined UNDO operation for each action that you want to roll back. Once you have that, your answer might be as simple as a try..catch, where the catch calls the undo operations. – Warmedover

Answer 1

The last time I saw something like this was several years ago. The little bit that I remember about it is that it was using the command pattern and storing each command object in a queue. I think it was a LIFO stack.

So if the "transaction" failed, the engine would pop off a command object, undo the command, then destroy the command object. Repeat until the stack was empty. The stack got cleared if the "transaction" was successful.

Unfortunately, I don't remember more than that.

CSLA.NET implements a similar undo stack. That's the only example with code that I can think off the top of my head.

Limonene

Answer 2

Windows Workflow Foundation has a concept of compensation (using Composite activity) when ACID semantics might not be appropriate..Off course, it has support for ACID transactions as well.

A good question is why bother with compensation? Isn’t one big ACID transaction with automatic rollback just as good? An ACID transaction is most appropriate when operations occur within the same database or within the same information system. It is also most appropriate when operations end quickly. When different companies and services are involved, defining the process in terms of the ACID semantics is often challenging. For it to be isolated and durable, you have to keep all resources of different companies locked for the duration of the task. This is frequently unreasonable, especially if the task is long. For it to be consistent and atomic, you need ad hoc compensation code.

Erasmoerasmus

Answer 3

The simplest technique without relying heavily on an external library is prevalence. Periodically checkpoint by using serialization to take a snapshot of your state, then maintain a journal by serializing enough information on every side-effectful operation against your data to repeat it later. If something blows up, reload the most recent checkpoint, then re-apply all journal records written after that point.

For something more sophisticated, try software transactional memory. It may be somewhat clumsy to implement in current languages, but is quite powerful and may give you some additional concurrency techniques as well.

For irreversible operations like accessing a Web service or sending an email, you'll need to use compensating transactions: make another Web service call to cancel or update the results of the previous one, or perhaps send another email advising the recipient that things didn't work as intended.

Hitchhike

Answer 4

Since you can't un-send an e-mail, and it's relatively inexpensive to write a file, I'd just do those things in the proper order:

1. Try to write the file/write the file. If unssuccessful, stop, otherwise continue to:
2. Call the web service. If unsuccessful, delete the file and stop, otherwise continue to:
3. Send e-mail -- email is asynchronous anyhow, so you'd never really know if it was sent or not since most e-mail servers are set to retry for a couple of days if an error occurs and you never get back an acknowledgment that the e-mail went through even if it was successful.

Middleweight

You may still be in trouble if the web service receives and processes the call but the response gets lost. I ran into this with eBay's AddItem call, which has the irreversible side effect of costing money. – Hitchhike

In the world of networking, never receiving the ACK is the same as a NACK, is it not? When this happens, often some manual intervention is necessary and unavoidable. – Middleweight

Answer 5

One idea is to use JMS as the 'engine' and you can utilize JMS transactions (which can join existing transactions e.g. DB transaction). This starts to lead towards an async event-driven architecture which is probably a good thing unless we are talking simple apps - in which case you probably don't need to be asking the question.

An example of this would be simple account creation. For this you want to persist account information to the DB and also send the user an email for activation - but you want them in the same transaction for obvious reasons.

You should not put email sending code within the transaction because even though you may send the email - db transaction commit may fail for one reason or another. You also should not put email sending outside of the transaction (after commit) because email sending may fail leading to an orphan account.

So to use JMS in this scenario - put JMS sending code within the DB transaction and have it join that transaction. You are guaranteed message delivery. On the other end have something consuming the queue sending emails out. In case of email send failure, best option is to log/raise an alert - JMS will roll back and put message back into the queue for later consumption. i.e. to try and resend email once you have hopefully fixed whatever the issue is.

The critical thing is - DB record is consistent and email is eventually sent.

Dimeter

Answer 6

Two thoughts:

• As Jeffrey Hantin mentions, software transactional memory (STM) is pretty sweet. Take a look at how it's implemented/used in Clojure; Rich Hickey's videos are impressive.
• One important aspect of the Command pattern of GoF fame is "undoability". An implementation of this might boil down to the simple (but nonetheless useful) proposal by Robert C. Barth

Death

Answer 7

Also, A experimental project STM .NET was recently released. This projects adds transaction memory to C#. It actually modifies the CLR to support this.

Mendy