Kafka Broker marks partitions as failed

Question

I notice the message in kafka broker logs

Partition <topic name> marked as failed (kafka.server.ReplicaFetcherThread)

My question is how and when does Kafka mark a partition as failed.
I need to monitor the logs and take action on messages.

Is this message actionable or something that can be ignored?

Sheerlegs

Answer 1

Kafka marks partitions as "Failed" whenever any issues with the replication process are encountered. The ReplicaFetcherThread is responsible for fetching data from leader replicas to replicate data to follower replicas. The issues could be anything, such as network, hardware, or storage. Kafka does this to ensure that the integrity of the data is maintained and to prevent data loss.

See: Understanding and Managing Failed Partitions in Kafka

KIP-461 - Improve Replica Fetcher behaviour at handling partition failure

Joinder