Disruptor: journaling Example

Question

I was curious regarding the most common (or recommended) implementations of disruptor about the journaling step. And the most common questions of mine are:

• how it is actually implemented (by example)?
• Is it wise to use JPA?
• What DB is commonly used (by the community that has already implement projects with disruptor)?
• Is it wise to be used at the intermediate handlers (of EventProcessors) so the State of each message should be saved, rather than before and after the business logic process?

By the way (I am sorry, I know this is not related with the journalling step), what is the right way to delete a message from the RingBuffer during an eventHandler process (assuming that the message is dead/expired and should be removed by the whole procedure). I was wondering something similar as the Dead Letter Channel pattern.

Cheers!

Lunar

Answer 1

The Disruptor is usually used for low latency, high throughput processing. E.g. millions of messages with a typical latency in the hundreds of micro-seconds. As very few databases can handle this sort of rate of updates with reasonably bounded delays, journaling is often done to a raw file with replication to a second (or third) system.

For reporting purposes, a system reads this file or listens to messages and updates a database as quickly as it can but this is taken out of the critical path.

An entry is dead in the ring buffer when every event processor has processed it.

---

The slot a message uses is not available until every event processor has processed it and all the message before it. Deleting a message would be too expensive, both in terms of performance and impact on the design

Every event processors sees every message. As this happens concurrently, there is little cost in doing this, but it quite normal for event processors to ignore messages as a result. (possibly most messages)

Energid

Yes the DB Reporting that I was referring it was assumed to be processed out of the Business critical path. And regarding the dead message, What if a message is expired during the process, it should be deleted (or forward it to a dead message repository). – Lunar

An expired message needs to be processed like any other message, its only what you do with it that differs e.g. you send an expired response instead of an accepted response. – Energid

An Expiration response is usually depended on other factors as well (on asynchronous architecture). But just to be clear, can't a message be removed from the RingBuffer before its completion? – Lunar

The slot a message uses is not available until every event processor has processed it and all the message before it. Deleting a message would be too expensive, both in terms of performance and impact on the design. – Energid

I see, that explains a lot regarding the message manipulation during the business logic process. thx! – Lunar

Every event processors sees every message. As this happens concurrently, there is little cost in doing this, but it quite normal for event processors to ignore messages as a result. (possibly most messages) – Energid

Indeed, I didn't include the cost of performance factor in that scale. Regarding the journaling step shouldn't be implemented by using a DB as a persistence layer? assuming that the Service (which use Disruptor) is resuming its process, this wouldn't effect overall performance to fetch Messages from a DB repository, and then start the RingBuffer to spin. – Lunar

The message would be fetched from the raw file rather than a DB which I imagine would be too slow. – Energid

BTW I implemented a different solution to processing messages which made persistent and replication the heart of the solution github.com/peter-lawrey/Java-Chronicle It can handle over ten million messages per second with very low latency and has almost no warmup time regardless of the number of messages you are behind. Your event processors can be in the same JVM or different processes with sub-microsecond latency. This allows you to restart an individual process with minimal impact on the overall system. – Energid

If nothing else you can use it for the journaling with the Disruptor. – Energid

I was just thinking of that, Where examples of your library can be found? – Lunar

In the unit tests. It pretty simple and supports the DataInput and DataOutput interfaces. You can also write text logs. Its designed to be lock less, mostly off heap and GC free. – Energid

it really seems Ideal, i'll keep up testing it but seems promising. thx! – Lunar

It performs well even if the consumer is more messages behind the producer than will fit in memory e.g. hundreds of millions. ;) – Energid

You should edit your answer to include your library and the answer regarding message deletion cost in performance. – Lunar

Since I wrote the library, I don't like self promotion. ;) – Energid