apache-kafka Questions
19
Solved
I've created docker with kafka broker and zookeeper to start it with run script.
If I do fresh start it starts normally and runs ok (Windows -> WSL -> two tmux windows, one session). If I shut down...
Omnifarious asked 4/1, 2020 at 16:14
2
Solved
I'm trying to configure a kafka client to authenticate against a secure kafkaserver. I've set up the jaas and ssl configs, but it's complaining about serviceNames.
I am not using Kerberos.
comman...
Wombat asked 4/3, 2019 at 18:49
2
I have a very strange problem when trying to connect locally to Kafka 0.10.0.0 using Python client on CentOS.
My connection options are pretty simple and default:
kafka_consumer = kafka.KafkaCons...
Corruptible asked 2/8, 2016 at 15:51
3
Solved
I see from the logs that exact same message is consumed by the 665 times. Why does this happen?
I also see this in the logs
Commit cannot be completed since the group has already rebalanced and a...
Appendicitis asked 7/8, 2018 at 23:4
4
Getting the following error (Kafka 2.1.0):
2018-12-03 21:22:37.873 ERROR 37645 --- [nio-8080-exec-1]
o.s.k.support.LoggingProducerListener : Exception thrown when
sending a message with key='n...
Selfregulated asked 4/12, 2018 at 3:27
14
I set on a ubuntu node of a cluster a kafka 0.11.0.0 instance.
Until some weeks ago everything worked fine, today I'm trying to starting it and I obtain this error after the boot:
[2017-09-11 16:2...
Hartebeest asked 11/9, 2017 at 14:36
2
I've written a python script with aiokafka to produce and consume from a Kafka cluster in AWS MSK, I'm running the script from a EC2 instance that is in the same VPC as my cluster and when I try to...
Estevez asked 5/6, 2022 at 14:58
9
Solved
I am trying to run a Kafka Streams application in kubernetes. When I launch the pod I get the following exception:
Exception in thread "streams-pipe-e19c2d9a-d403-4944-8d26-0ef27ed5c057-StreamThre...
Linstock asked 11/5, 2018 at 8:19
6
Solved
To get Kafka running, you need to set some properties in config/server.properties file. There are two settings I don't understand.
Can somebody explain the difference between listeners and advert...
Guarnerius asked 24/3, 2017 at 11:54
8
Kafka Version : 0.10.2.1,
Kafka Producer error Expiring 10 record(s) for TOPIC:XXXXXX: 6686 ms has passed since batch creation plus linger time
org.apache.kafka.common.errors.TimeoutException: Expi...
Forme asked 14/10, 2017 at 23:54
3
Solved
I have one Kafka topic and five partitions for that one topic. There will be 5 consumer groups. Each consumer group has one service instances consuming from that topic.
Will the offset be the same...
Lurid asked 17/12, 2018 at 15:49
2
I have a topic which is read as GlobalKTable and Materialized in a store. The issue is if I update a key on the topic and then read from store, for a while(~0.5sec) I get the old value.
What could...
Winton asked 9/1, 2019 at 6:49
11
I enabled JMX on Kafka brokers by adding
KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote=true
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Djava.r...
Aerify asked 19/4, 2016 at 4:8
13
Solved
Below the steps I did to get this issue :
Launch ZooKeeper
Launch Kafka : .\bin\windows\kafka-server-start.bat .\config\server.properties
And at the second step the error happens :
ERROR F...
Melda asked 25/12, 2019 at 21:10
4
Solved
Spark 2.2 introduced a Kafka's structured streaming source. As I understand, it's relying on HDFS checkpoint directory to store offsets and guarantee an "exactly-once" message delivery.
But old do...
Dorsey asked 11/9, 2017 at 10:7
7
Solved
I am running in my locahost both Zookeeper and Kafka (1 instance each).
I create succesfully a topic from kafka:
./bin/kafka-topics.sh --zookeeper localhost:2181 --create --replication-factor 1 -...
Harvin asked 6/12, 2017 at 15:12
3
I am using Kafka for Event Sourcing and I am interested in implementing sagas using Kafka.
Any best practices on how to do this? The Commander pattern mentioned here seems close to the architecture...
Vo asked 8/5, 2017 at 10:21
4
I am working on a use case where I have created pipeline which sends data from mongo to elasticsearch.
Mongo -> Spring Boot -> Kafka -> Transformer(KStream) -> Kafka -> Consumer (Send to Elastic S...
Although asked 27/4, 2019 at 17:2
6
Solved
I have the consumer config as follows
package com.example.kafka.config;
import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.common.serialization.StringDeserializer;
im...
Sialkot asked 21/7, 2022 at 14:33
3
Solved
I am learning Kafka and trying to use it with docker. I'm confused looking at the docker-compose files, so I wanted to ask my questions here.
In most examples, I see a config like this:
broker:
im...
Sup asked 8/10, 2020 at 4:42
7
I have the following folder structure:
from C:\Program Files\kafka\kafka_2.11-0.11.0.0 frolder I try to execute following command:
bin\windows\zookeeper-server-start.bat config\zookeeper.proper...
Heterosporous asked 1/9, 2017 at 13:35
3
Solved
I am running a data pipeline such that
I read data from sql db into kafka topic through jdbc connect
I sink this data in Elasticsearch using kafka sink connector for ES
I have a need to reset thi...
Diapositive asked 23/10, 2018 at 20:29
17
Solved
Is there a way to delete all the data from a topic or delete the topic before every run?
Can I modify the KafkaConfig.scala file to change the logRetentionHours property? Is there a way the messag...
Roybn asked 18/7, 2013 at 18:8
1
I have my consumer setup, I can post the message successfully from the producer app.
here is my producer service method
@Service
public class KafkaMessagePublisher {
private final KafkaTemplate<...
Juliannajulianne asked 12/1 at 2:52
2
Solved
I have a kafka stream application in which it is using stateStore (backed by RocksDB).
All what stream thread is doing is getting data from kafka topic and putting the data into state-store. (The...
Jesseniajessey asked 1/5, 2020 at 21:52
© 2022 - 2024 — McMap. All rights reserved.