XA/JTA transaction: JMS message arrives before DB changes are visible

Question

Context is:

• producer (JTA transaction PT) is both sending message to JMS queue and making DB update;
• consumer (JTA transaction CT) listens on same queue and reads DB when message is received;
• application server - WebLogic, DB - Oracle.

I've observed, that sometimes CT is not (yet?) able to see DB changes of PT, event if corresponding JMS message is already received (PT is committed?).

It seems that JTA can't guarantee consistency of such kind (this was also confirmed in Jurgen Holler's presentation "Transaction Choices for Performance").

What is the best way to avoid such problem (except obvious - not using JTA)?

Thanks.

Promethean

Just an as another idea: since Weblogic & Oracle are alrady being used... you might as well add in Coherence (a distributed in-memory cache for the database). You update the cache at the same time as the database (a write-through cache), so the cache has the latest value immediately available to your consumer (or at least with decreased latency). – Agog

Answer 1

So it seems there is no simple, elegant and fail-proof solution for that. In our case it was decided to rely on simple redelivery mechanism (throwing exception and letting JMS message to be redelivered after certain amount of time).

Also considered:

• Marking DB datasource as and expecting Last Resource Commit Optimization (LRCO) to kick-in (thus partially controlling order of commits inside XA transaction). Rejected due to dependency to internals of application server (WL).

• Setting DeliveryDelay to JMS message, so it can be consumed only after some time, when (supposedly) DB sync is over. Rejected due to lack of guarantee and need to fine-tune it for different environments.

Blog post mentioned in other answer indeed contains all these and several other options covered (but no definitive one).

Promethean

I guess the best solution would be to enable both redeliviery mechanism and a small delivery delay so that the need of redeliviery occurs only in extreme cases. – Amos

Answer 2

some options are outlined here: http://jbossts.blogspot.co.uk/2011/04/messagingdatabase-race-conditions.html

Humbert

Answer 3

Concerning the Answer:

"So it seems there is no simple, elegant and fail-proof solution for that. In our case it was decided to rely on simple redelivery mechanism (throwing exception and letting JMS message to be redelivered after certain amount of time)."

This is only fail proof if your second transaction that starts after Transaction1 logically ends has a way of detecting that the Transaction 1 changes are not yet visible and blow up itself on techichal exception.

When you have Transaction 2 that is a different process than Transaction 1 then this is likely to be possible to check. Most likely the output of Transaction 1 is necessary to the success of transaction 2 to go forward. You can only make french fries if you have potatoes... If you have no potatoes you can blow up and try again next time.

However, if your process that is breaking due to the DB appearing stale is the exact same process that run on Transaction 1 itself. You are just adding potatoes into a bowel (e.g. a db table) and fail to detect that you bowel is overlfowing and continue running transactions to pumptit up... Such a check may be out of your hands.

Something of the sort, happens to be my case.

A theoretical solution for this might very well be to try to induce a Dirty Read on the DB by creating an artificial entity equivalent to the @Version field of JPA, forcing each process that needs to run serially to hammer an update on a common entity. If both transaction 2 and 1 update a common field on a common entity, the process will have to break - either you get a JPA optimistic lock exception on the second transaction or if you get a dirty read update exception from the DB.

I have not tested this approach yet, but it is likely going to be the needed work around, sadly enough.

Wildebeest

I have tested the approach of the read write same entity forcing the read and write reach the datbase, and fixes the problem. We can be assured to logical serialization of business logic with this approach. – Wildebeest