SAGA hints the issue:
There are also the following issues to address:
...
- In order to be reliable, a service must atomically update its database and publish an event. It cannot use the traditional mechanism of a distributed transaction that spans the database and the message broker. Instead, it must use one of the patterns listed below.
...
The following patterns are ways to atomically update state and publish events:
- Event sourcing
- Application events
- Database triggers
- Transaction log tailing
Event Sourcing is special in this list as it brings radical change on how your system stores and processes data. Usually, systems store only the current state of the entities. Some systems add explicit support for historical states with validity periods and/or bitemporal data.
Systems which are based on Event Sourcing store the sequence of events instead of entity state in a way that allows it to reconstruct the state from events. There is only one transactional resource to maintain - event store - so there is no need to coordinate transactions.
Other patterns in the list avoid the issue of transaction coordination by requiring the event producer code to commit all changes - both entities state and events (as entities) - to the single data store. Then a dedicated, but separate mechanism - event publisher - is implemented to fetch the events from the data store and publish them to the event consumers.
Event publisher would need to keep the track of published / unpublished events which usually brings back the problem of coordinated transactions. That's were the idempotency of the event consumers comes to light. Event publisher will replay events from the last known position while consumers will ignore duplicates.
You may also reverse the active / passive aspects of the event producer and event consumer. Event producer stores entities state and events (as entities) to the single data store and provides an endpoint which allows event consumer to access event streams. Each event consumer keeps track of processed / unprocessed events - which it needs to do anyway for idempotency reasons - but only for event streams it is interested about. A really good explanation of this approach is given in the book REST in Practice - chapters 7 and 8.
