I am sending an array of JSON by converting it to toString() in Kafka Producer using Spring Boot app, but I am getting following error in Consumer: org.springframework.kafka.listener.ListenerExe...

I have two topics, one with 3 partitions and one with 48. Initially i used the default assignor but i got some problems when a consumer(pod in kubernetes) crashed. What happened was that when t...

The description of the Kafka topic cleanup.policy configuration is A string that is either "delete" or "compact" or both. [...] I wonder how to set both values. I am not able ...

I need to automate a rolling restart of a kafka cluster (3 kafka brokers). I can easily do it manually - restart one after the other, while checking the log to see when it's fine (e.g., when the ne...

i am trying to create producer and send some data to it from intellij idea but while running this program i got ERROR like ERROR org.apache.kafka.common.utils.KafkaThread - Uncaught exception in th...

I have this exception in the consumer when trying to cast the record.value() into java object : ClassCastException: class org.apache.avro.generic.GenericData$Record cannot be cast to class [...].Pu...

I have a spring boot (2.1.6) application that both consumes and produces messages to a (organization wide) common kafka instance. I am trying to implement health checks for this kafka broker in my ...

Running into issues when trying to use kerberos auth with connecting to Kafka. Using scala and my jaas.config looks something like this. KafkaClient { com.sun.security.auth.module.Krb5LoginModul...

I don't understand the practical use case of the consumer group in Kafka. A partition can only be read by only one consumer in a consumer group, so only a subset of a topic record is read by one co...

I have Kafka commit policy set to latest and missing first few messages. If I give a sleep of 20 seconds before starting to send the messages to the input topic, everything is working as desired. I...

Other than using JMX is there any other way to know, whether a broker is an ActiveController? I know that the cluster generates a metric kafka.controller:type=KafkaController,name=ActiveController...

I'm trying to figure out the difference between the settings batch.size and buffer.memory in Kafka Producer. As I understand batch.size: It's the max size of the batch that can be sent. The docum...

We're trying to use kafka streams for our project to read data from one topic and write to another, and we have a use case to use KafkaHeaders as a mechanism to filter our certain records. Example,...

I have configured kafka_2.11-2.3.0 and apache-zookeeper-3.5.5-bin on Windows 10. But while running the topic creation command I am getting the below error: C:\kafka_2.11-2.3.0>.\bin\windows\kaf...

I have been working with Kafka 2.4.0 (2.11) and yesterday I had to forcefully terminate the process for some unknown reason. Since then I haven't been unable to start Zookeeper due to the following...

I was able to successfully set up zookeeper and one kafka broker yesterday. Everything worked as expected. I shut down kafka (ctrl + c) and then zookeeper. Today I started zookeeper and when I sta...

This is the extension that I am trying to install: https://github.com/EVODelavega/phpkafka The messages passed to the queue should be in JSON format. Currently, I am getting installation errors...

I'm setting up a managed kafka queue on AWS MSK. I can't seem to get the security to work when connecting from a local machine and I can't work out if I can use security groups from one region to a...

Is this any bug with the Apache Kafka 0.9.0.0? I'm using the following command to list the topics and I get nothing even though the server was started with a topic being created! Joes-MacBook-Pro:...

Environment 3-node Kafka Cluster Amazon MSK v2.3 1 topic 6 partitions 1 consumer group with 2 consumers Running in Kubernetes Confluent .NET SDK 1.2.2 Except for bootstrap.servers and group...

I need to use the Confluent kafka-avro-serializer Maven artifact. From the official guide I should add this repository to my Maven pom <repository> <id>confluent</id> <url&g...

I'm learning about Microservice Architecture these days and I need to run Kafka to follow some tutorials. However, zookeeper-server-start on cmd which is the first step for running up Kafka doesn't...

I'm following this tutorial trying to set up kafka, https://kafka.apache.org/quickstart And I get this error when doing the part on zookeeper. It certainly has something to do with the config, b...

How to print response from Kafka console consumer to JSON pretty. I have tried this but it's not working. ./kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic indexing-v1 --from-...

I notice the message in kafka broker logs Partition <topic name> marked as failed (kafka.server.ReplicaFetcherThread) My question is how and when does Kafka mark a partition as failed. I need...