kafka-python Questions
12
I am trying to use wurstmeister\kafka-docker image with docker-compose, but I am having real problems with connecting everything.
All the posts or questions that I check, seems not to have any pro...
Whomp asked 8/3, 2016 at 7:31
2
I have a very strange problem when trying to connect locally to Kafka 0.10.0.0 using Python client on CentOS.
My connection options are pretty simple and default:
kafka_consumer = kafka.KafkaCons...
Corruptible asked 2/8, 2016 at 15:51
4
Solved
I have used Kafka Streams in Java. I could not find similar API in python.
Do Apache Kafka support stream processing in python?
Ladylike asked 19/8, 2018 at 14:59
2
I have a use case of high throughput kafka producer where I want to push thousands of json messages every second.
I have a 3 node kafka cluster and I am using latest kafka-python library and have ...
Spermous asked 4/6, 2020 at 15:37
3
Solved
I am using python-kafka to listen to a kafka topic and use that the records. I want to keep it polling infinitely without any exit. This is my code below:
def test():
consumer = KafkaConsumer('abc...
Lilalilac asked 8/2, 2021 at 12:15
4
Solved
I have following program to consume all the messages coming to Kafka.
from kafka import KafkaConsumer
consumer = KafkaConsumer('my_test_topic',
group_id='my-group',
bootstrap_servers=['my_kafka:...
Hube asked 17/7, 2017 at 12:26
6
Solved
Using kafka-python-1.0.2.
If I have a topic with 10 partitions, how do I go about committing a particular partition, while looping through the various partitions and messages. I just cant seem fin...
Christiansen asked 12/4, 2016 at 17:8
8
Solved
I am using the Python high level consumer for Kafka and want to know the latest offsets for each partition of a topic. However I cannot get it to work.
from kafka import TopicPartition
from kafka...
Lovett asked 16/2, 2016 at 12:13
7
Solved
i have already started to learn Kafka. Trying basic operations on it. I have stucked on a point which about the 'Brokers'.
My kafka is running but when i want to create a partition.
from kafka ...
Aleedis asked 9/8, 2016 at 15:34
2
I am using kafka-python for accessing Kafka. I try to create a Kafka Producer:
kafka_producer = KafkaProducer(bootstrap_servers=['kafka:9092'])
but this fails with exception kafka.errors.NoBroke...
Sechrist asked 17/7, 2019 at 13:15
4
As said in the title, i want to get a number of record in my topic and i can't find a solution using kafka-python library.
Does anyone have any idea ?
Glynda asked 7/10, 2020 at 15:46
7
I am having trouble with KafaConsumer to make it read from the beginning, or from any other explicit offset.
Running the command line tools for the consumer for the same topic , I do see messages...
Violaceous asked 5/2, 2016 at 6:28
3
I am using Kafka 2.12 and kafka-python module as Kafka client. I am trying to test a simple producer:
class Producer(Process):
daemon = True
def run(self):
producer = KafkaProducer(bootstrap_serv...
Floccose asked 14/7, 2017 at 0:33
4
Solved
I'm trying to create kafka producer with ssl. I need information on how to set SSL parameters in the constructor, the information provided in kafka-python client is not descriptive enough.
What a...
Overlie asked 22/8, 2018 at 3:33
4
Solved
I am doing Python Kafka consumer (trying to use kafka.consumer.SimpleConsumer or kafka.consumer.simple.SimpleConsumer in http://kafka-python.readthedocs.org/en/latest/apidoc/kafka.consumer.html). W...
Woofer asked 5/8, 2015 at 19:27
3
Solved
I am using Kafka (kafka-python) version 3.0.0-1.3.0.0.p0.40. I need to configure the consumer for the topic 'simulation' in Python. When I don't indicate the group_id, i.e. group_id = None it recei...
Didynamous asked 21/9, 2018 at 0:48
1
I am trying to gracefully shutdown a kafka consumer, but the script blocks with Stopping HeartBeat thread.
How can i gracefully close the consumer on a SIGTERM with kafka-python.
This is what i hav...
Hedberg asked 28/4, 2020 at 7:21
1
I'm somewhat new to integration tests. I have two services that pass messages to one another using Kafka. However, for my integration tests, I don't necessarily want to have Kafka running in order ...
Johiah asked 25/11, 2016 at 7:52
3
Solved
We have a message scheduler that generates a hash-key from the message attributes before placing it on a Kafka topic queue with the key.
This is done for de-duplication purposes. However, I am not...
Watts asked 31/10, 2016 at 10:9
2
Solved
I am getting this error when constructing KafkaProducer with the kafka-python package:
[ERROR] UnrecognizedBrokerVersion: UnrecognizedBrokerVersion
Traceback (most recent call last):
File "/var/l...
Kalevala asked 31/10, 2019 at 9:15
1
Solved
Traceback (most recent call last):
File "//producer.py", line 1, in <module>
from kafka.producer import KafkaProducer
File "/usr/local/lib/python3.9/site-packages/kafka/__in...
Mighty asked 20/1, 2021 at 12:21
2
Solved
I'm subscribing to Kafka using a pattern with a wildcard, as shown below. The wildcard represents a dynamic customer id.
consumer.subscribe(pattern='customer.*.validations')
This works well, bec...
Lindsy asked 15/9, 2016 at 20:56
2
Solved
(Possibly a duplicate of Can't send a keyedMessage to brokers with partitioner.class=kafka.producer.DefaultPartitioner, although the OP of that question didn't mention kafka-python. And anyway,...
Scorpio asked 2/8, 2017 at 20:9
1
Solved
How to delete quotes and send data like on original format
The original JSON-format is:
{
"@timestamp": "2020-06-02T09:38:03.183186Z"
}
This data in another topic
"{\"@timestamp\": \"2020-05-2...
Spoor asked 2/6, 2020 at 9:49
0
I am trying to produce the messages to a Kafka topic using kafka-python 2.0.1 using python 2.7 (can't use Python 3 due to some workplace-related limitations)
I created a class as below in a separa...
Effortful asked 8/5, 2020 at 9:45
1 Next >
© 2022 - 2024 — McMap. All rights reserved.