CQRS: Storing events and publishing them - how do I do this in a safe way?

Question

As I've learned in Why is the CQRS repository publishing events, not the event store? it's the CQRS repository's task to publish events. So far, so good.

Of course, storing the events and publishing them should be within one single transaction. Technically this means writing one (or more) records to the store, and publishing one (or more) events to a message bus. Hence, a simple database transaction is not enough, it should be a distributed one.

Now, unfortunately, many NoSQL databases (such as MongoDB) do not support ACID-compliant transactions, not even to talk of the possibility of taking place in a distributed transaction. More over, there are message queues out there that do not support distributed transactions as well.

So the question is: How do I deal with this?

Is there a recommended pattern to use?

As

Answer 1

Your repository could publish events, it doesn't have to. The solution in this case is to use the event store as a queue. You would have a background process that monitors the event store for new events, publishes them to (for instance) a bus, and then marks them as dispatched.

As always, there are trade-offs. You'll likely have to deal with at-least-once messaging and idempotent processing. It's more complex than using a simple distributed transaction.

Jonathan Oliver has written several posts about this topic that might help you out: Removing 2PC, How I Avoid Two-Phase Commit, Idempotency Patterns

Cringe

Thank you for this - incredibly helpful. Regarding monitoring of the event store - how could one accomplish this without polling? Is there a more responsive mechanism to do this while not introducing 2PC? – Sechrist