How continuous Azure Web Jobs can be idempotent and send email?

Question

After reading tons of information on the web about Azure WebJobs, documentation says a job should be idempotent, on the other hand, blogs say they use WebJobs to do actions such as "charging a customer", "sending an e-mail".

This documentation says that running a continuous WebJob on multiple instances with a queue could result in being called more than once. Do people really ignore the fact that they could charge their customer twice, or send an e-mail twice?

How can I make sure I can run a WebJob with a queue on a scaled web app and messages are processed only once?

Fivepenny

Answer 1

I do this using a database, an update query with a row lock and TransactionScope object.

In your Order table, create a column to manage the state of the action you are taking in your WebJob. i.e. EmailSent.

In the QueueTrigger function begin a transaction, then execute an UPDATE query on the customer order with ROWLOCK set, that sets EmailSent = 1 with a WHERE EmailSent = 0. If the return value from SqlCommand = 0 then exit the function. Another WebJob has already sent the email. Otherwise, send the email and call Complete() on the TransactionScope object if sent successfully.

That should provide the idempotency you want.

Hope that helps.

Knitted

I thought of this as my last solution. However, how are the failures managed? If one of the server completes with no failure and no e-mail sent, then the other server does not complete and has error, how will the failure be managed here? Will it re-trigger? Go to poison? or completely ignore the error? I feel this is a half done feature from Microsoft. The lock should be implemented at the SDK level. – Fivepenny

I know this is an old question but you can additionally add a timestamp to that table so you can log when it gets locked so if that timestamp is more than X minutes old, assume it failed and clear it to allow the next message to process it. You are right that this isn't fully implemented in the SDK but this isn't something that the SDK itself can handle since every situation is different. I mean, what if you don't have a SQL Server database? You obviously have to have a different solution. And how is the SDK to know what the scope is of idempotency? Again, this changes on a case-by-case basis. – Soppy