kafka-consumer-api Questions
4
I am using Spring Kafka first time and I am not able to use Acknowledgement.acknowledge() method for manual commit in my consumer code as mentioned here https://docs.spring.io/spring-kafka/referenc...
Butte asked 20/9, 2017 at 15:1
2
Solved
I am building a kafka manager tool and I need to check which topic-partition is assigned to which consumer in a consumer group.
Suppose there is consumer-Group group-A consuming topic topic-A with...
Auerbach asked 6/9, 2016 at 11:12
6
Solved
Using kafka-python-1.0.2.
If I have a topic with 10 partitions, how do I go about committing a particular partition, while looping through the various partitions and messages. I just cant seem fin...
Christiansen asked 12/4, 2016 at 17:8
2
I've just deployed my Kafka Connect (I only use a connect source to
MQTT) application on a cluster of two instances (2 containers on 2
machines) and now it seems to get into a sort of rebalancing l...
Hurter asked 11/8, 2017 at 12:54
5
Solved
I am starting to learn Kafka. During my readings, some questions came to my mind:
When a producer is producing a message, it will specify the topic it wants to send the message to. Is that right? ...
Mickel asked 25/6, 2016 at 2:58
9
Solved
I need to find out a way to ask Kafka for a list of topics. I know I can do that using the kafka-topics.sh script included in the bin\ directory. Once I have this list, I need all the consumers per...
Scion asked 21/9, 2015 at 14:38
1
Solved
I am using kubernetes microserice in which I have create single Kafka micro service to consume all topic and perform its activities. Code sample below:
const { Kafka, logLevel } = require("kaf...
Irresistible asked 20/5, 2023 at 5:17
8
I have to add encryption and authentication with SSL in kafka.
This is what I have done:
Generate certificate for each broker kafka:
keytool -keystore server.keystore.jks -alias localhost -validi...
Langill asked 27/2, 2019 at 10:32
5
Consider the Kafka Producer :
const { Kafka, logLevel } = require('kafkajs');
const kafka = new Kafka({
clientId: 'my-app',
brokers: ['kafka1:9092', 'kafka2:9092'],
logLevel: logLevel.ERROR,
})...
Plating asked 9/10, 2020 at 8:27
2
Here's what I've tried so far:
from confluent_kafka import Consumer
c = Consumer({... several security/server settings skipped...
'auto.offset.reset': 'beginning',
'group.id': 'my-group'})
c.su...
Giant asked 8/2, 2021 at 21:37
2
Solved
9.0 client to consume messages from two brokers which are running on a remote system.My producer is working fine and is able to send messages to the broker but my consumer is not able to consume th...
Latrice asked 1/6, 2016 at 9:34
3
Solved
Whenever I am trying to read the message from kafka queue, I am getting following exception :
[error] (run-main-0) java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record cannot ...
Dewyeyed asked 13/2, 2017 at 9:38
8
Solved
I am using the Python high level consumer for Kafka and want to know the latest offsets for each partition of a topic. However I cannot get it to work.
from kafka import TopicPartition
from kafka...
Lovett asked 16/2, 2016 at 12:13
7
Solved
i have already started to learn Kafka. Trying basic operations on it. I have stucked on a point which about the 'Brokers'.
My kafka is running but when i want to create a partition.
from kafka ...
Aleedis asked 9/8, 2016 at 15:34
3
Solved
Kafka returns endOffset 15 for a partition, but the last message that can be consumed from has the offset 13, rather than 14, which I would expect. I wonder why.
The Kafka docs read
In the default...
Ruthy asked 15/9, 2021 at 2:8
0
I have a go project with confluent Kafka and I want to change it's initial connection timeout. If the kafka is not available on the first try I want to retry connecting to it every few seconds and ...
Paraprofessional asked 8/12, 2022 at 10:45
3
Solved
We have an HDP cluster with 3 kafka brokers ( from hortonworks )
We want to run kafka console consumer in order to get one message from topic with specific offset
/usr/hdp/current/kafka-broker/b...
Kaenel asked 11/12, 2019 at 17:21
4
I am working on a kafka consumer program. Recently we deployed it in PROD environment. There we faced an issue as follows:
[main] INFO com.cisco.kafka.consumer.RTRKafkaConsumer - No. of records fet...
Goodtempered asked 30/3, 2020 at 5:59
8
I'm working with Apache Kafka and its Java client and I see that messages are load balanced across different Kafka Consumers belonging to the same group (i.e. sharing the same group id).
In my app...
Pentagram asked 30/3, 2017 at 13:1
4
I am using Confluent kafka C# client. how to get the latest offset consumed from a topic in this?
Pectoralis asked 8/7, 2017 at 7:1
4
I'm using KafkaConsumer to consume messages from Kafka server (topics)..
It works fine for topics created before starting Consumer code...
But the problem is, it will not work if the topics cr...
Telium asked 22/3, 2016 at 11:56
1
Kafka consumer application is throwing ObjectDisposedException when we initiate the process to stop the consumer service.I would like to understand what process destroyed the kafka handle
System.Ob...
Affiche asked 12/8, 2020 at 22:56
4
am getting below exception while connecting Event Hub via kafka libraries.
Caused by: org.apache.kafka.common.protocol.types.SchemaException: Error reading field 'sasl_auth_bytes': Bytes size -1 ...
Study asked 24/1, 2020 at 5:58
4
I am using Kafka with Spring-boot:
Kafka Producer class:
@Service
public class MyKafkaProducer {
@Autowired
private KafkaTemplate<String, String> kafkaTemplate;
private static Logger...
Meteorite asked 9/10, 2017 at 15:15
3
We have 10 consumers in a group listening for a topic. What is happening very often is to see the consumers being rebalanced very often (which completely stops the consumer process for some time).
...
Insanitary asked 18/6, 2019 at 10:50
© 2022 - 2024 — McMap. All rights reserved.