Does Kafka supports XA transactions?

Question

Can JDBC transactions and kafka transactions work together as XA transaction? Does kafka transaction API supports XA transaction?

Manatee

Not sure what you mean. Kafka doesn't have a JDBC driver – Virgievirgil

I am not asking about kafka JDBC driver. Kafka has its own transaction API - cwiki.apache.org/confluence/display/KAFKA/… – Manatee

Yes, I'm aware. I'm asking about how a JDBC transaction would work together with Kafka if it doesn't have any JDBC protocol support – Virgievirgil

Kafka need not have to support JDBC protocol. My question is whether kafka provides support for XA transactions with 2 phase commit. JDBC is just one example participant, it could be any other system\platform which supports XA transactions. Please refer - en.wikipedia.org/wiki/X/Open_XA – Manatee

Answer 1

Kafka has no support for XA transactions. Kafka delegates the burden of exactly-once to the message consumer.

Tidemark

Answer 2

Looking at the "Unsupported Features" section of

https://docs.confluent.io/platform/current/clients/kafka-jms-client/index.html

Transaction (XA or otherwise) are not supported by Kafka.

Both the Producer and the Consumer need to boilerplate an ad-hoc transaction manager 80's style.

I know there is lots of rhetoric floating around in the Spring and Microservices camps that claim everything from Transactions are impossible under the CAP Theorem to Distributed Transactions are proprietary vendor lock-in. And there seems to be a lot of confusion where people think transactions are just a Database thing.

But the reality is we have had interoperable, standardized and opensource XA on remote calls, including both databases, message services, and more since the early 90's

To be honest I don't look at Kafka as a product ready for production use it simply doesn't have the foundational features to support non-trivial message applications. And performance seems to be linked to cranking up the cloud hosting bill.

Listing

Answer 3

There's a missing piece of API to support XA transactions: the ability to resume a transaction after the producer reconnects. Some systems have hacked a way around it, you can have a look here: https://github.com/hazelcast/hazelcast/blob/master/extensions/kafka/src/main/java/com/hazelcast/jet/kafka/impl/ResumeTransactionUtil.java.

However, the API is not X/Open standard, but can do the job.

Bullhead