app engine datastore transaction exception

Question

In app engine transactions documentation I have found the following note:

Note: If your app receives an exception when submitting a transaction, it does not always mean that the transaction failed. You can receive Timeout, TransactionFailedError, or InternalError exceptions in cases where transactions have been committed and eventually will be applied successfully. Whenever possible, make your Datastore transactions idempotent so that if you repeat a transaction, the end result will be the same.

This is quite general information and I wasn't able to find more details. I have the following questions regarding this issue:

1. Does it affect NDB transations? NDB documentation doesn't mention it, but I suppose that this behavior is inherited
2. What can cause this type of situation?
3. How often can it happen?
4. Can I prevent it, or decrease probability?
5. Are transactional tasks enqueued in this situation?
6. Is this situation a bug, which will be fixed in the future, or a feature, which I should just get used to?

Dhow

Transactions are not really a feature of nosql data structures - look at this answer https://mcmap.net/q/241538/-transactions-in-nosql – Shanahan

The key is making it idempotent. Then it does not matter if it's repeated. – Depurate

I'm relatively new in this area, but it doesn't seem to be that easy. Sometimes I need to enforce uniqueness - think user name for example. If I get an exception I can retry the transaction, but what to do if it succeeded. I won't know who created the record - me or some other thread. I just need more knowledge before I design the right solution – Dhow

Answer 1

Yes, it affects ndb too.

Potential causes include network partitions where the datastore server commits successfully but cannot communicate the result to the app.

It is rare, but cannot be prevented, and will never be fixed. It is inherent to all distributed systems.

Task queue adds are committed with the transaction by the datastore server.

Cappadocia