Spring KafkaListener: How to know when it's ready

Question

I have a simple Spring Boot application which reads from Kafka and writes to Kafka. I wrote a SpringBootTest using an EmbeddedKafka to test all that.

The main problem is: Sometimes the test fails because the test sends the Kafka message too early. That way, the message is already written to Kafka before the Spring application (or its KafkaListener to be precise) is ready. Since the listener reads from the latest offset (I do not want to change any config for my test - except bootstrap.servers), it will not receive all messages in that test.

Does anyone have an idea how I could know inside the test, that the KafkaListener is ready to receive messages?

Only way I could think of is waiting until /health comes available but I have no idea whether I can be sure that this implies the KafkaListener to be ready at all.

Any help is greatly appreciated!

Best regards.

Myatt

Answer 1

If you have a KafkaMessageListenerContainer instance, then it is very easy to use org.springframework.kafka.test.utils.ContainerTestUtils.waitForAssignment(Object container, int partitions).

https://docs.spring.io/spring-kafka/api/org/springframework/kafka/test/utils/ContainerTestUtils.html

e.g. calling ContainerTestUtils.waitForAssignment(container, 1); in your Test setup will block until the container has gotten 1 partition assigned.

Heuer

Answer 2

So, I just read about @PostConstruct and it turns out that you can easily use this also within the test:

@PostConstruct
public void checkApplicationReady() {
    applicationReady = true;
}

Now I added an @Before method to wait until that flag is set to true.

So far this seems to work very nicely!

Myatt

Nope, the issue still occurs. – Myatt

Current idea is implementing ConsumerSeekAware and setting a boolean when onPartitionsAssigned was called. This is stable now (at least after ~100 runs) – Myatt