apache-kafka-streams Questions
2
Solved
I have two topics, one with 3 partitions and one with 48.
Initially i used the default assignor but i got some problems when a consumer(pod in kubernetes) crashed.
What happened was that when t...
Moonset asked 16/4, 2019 at 14:45
2
Solved
We're trying to use kafka streams for our project to read data from one topic and write to another, and we have a use case to use KafkaHeaders as a mechanism to filter our certain records.
Example,...
Highams asked 19/7 at 12:4
2
I want to give some time to my consumer to restart so that unnecessary rebalance doesnt happen. How can I do that?
In case of shutdown, I want replication to come in picture and after some time if ...
Imphal asked 12/6, 2019 at 11:40
1
The requirement is to send alerts if the expected 'final' event (identified from the evenType field in event payload) is not received for a key within a time window of say 2 minutes on the input to...
Lepsy asked 28/2, 2021 at 20:51
9
Solved
I am trying to run a Kafka Streams application in kubernetes. When I launch the pod I get the following exception:
Exception in thread "streams-pipe-e19c2d9a-d403-4944-8d26-0ef27ed5c057-StreamThre...
Linstock asked 11/5, 2018 at 8:19
2
I have a topic which is read as GlobalKTable and Materialized in a store. The issue is if I update a key on the topic and then read from store, for a while(~0.5sec) I get the old value.
What could...
Winton asked 9/1, 2019 at 6:49
1
I'm trying to test a KafkaStreams application.Using JUnit5 and EmbededKafka.
When executing any TestCase, I see that the logs are bombarded with the following messages
[Producer clientId=myTask-227...
Creationism asked 3/2, 2022 at 15:10
4
I am working on a use case where I have created pipeline which sends data from mongo to elasticsearch.
Mongo -> Spring Boot -> Kafka -> Transformer(KStream) -> Kafka -> Consumer (Send to Elastic S...
Although asked 27/4, 2019 at 17:2
2
Solved
I have a kafka stream application in which it is using stateStore (backed by RocksDB).
All what stream thread is doing is getting data from kafka topic and putting the data into state-store. (The...
Jesseniajessey asked 1/5, 2020 at 21:52
4
Solved
I have used Kafka Streams in Java. I could not find similar API in python.
Do Apache Kafka support stream processing in python?
Ladylike asked 19/8, 2018 at 14:59
3
Has anyone managed to debug kafkastreams code written in Java 8 using IntelliJ IDEA?. I am running a simple linesplit.java code where it takes stream from one topic and splits it and sends it to an...
Joris asked 20/6, 2018 at 18:6
7
Solved
I'm writing a Kafka Streams application on my development Windows machine.
If I try to use the leftJoin and branch features of Kafka Streams I get the error below when executing the jar application...
Linettelineup asked 2/5, 2017 at 16:1
5
I'm using following properties for Log4j:
//log4j.properties
log4j.rootLogger=DEBUG, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.ap...
Hornbook asked 24/5, 2018 at 12:40
1
Is there a way to limit or define the max memory usage of a kafka streams application? I have enabled caching with my state stores but when I deploy in Openshift I get OOM killed on my pods. I have...
Zerla asked 29/4, 2019 at 18:56
2
Solved
I'm developing a PoC with Kafka Streams. Now I need to get the offset value in the stream consumer and use it to generate a unique key (topic-offset)->hash for each message. The reason is: the p...
Beebread asked 25/11, 2016 at 14:38
2
Solved
I am looking for a way to merge two Kafka topics based on the event time.
for example, I have two topics with the following schema {event-key}:: {event-time-as-value}
topic I - { {1 :: 12:00pm} ...
Skvorak asked 4/9, 2019 at 11:45
5
A brief explanation of what I want to achieve:
I want to do functional tests for a kafka stream topology (using TopologyTestDriver) for avro records.
Issues: Can't "mock" schemaRegistry to automa...
Tusker asked 10/10, 2018 at 9:44
3
I have implemented kafka stream application. Let's say one of the object's field which the stream is currently processing contains a number instead of a string value. Currently when there is an exc...
Sundberg asked 27/1, 2022 at 17:25
2
Solved
I think it's related to the below links, but I don't understand.
https://issues.apache.org/jira/browse/KAFKA-6535
https://issues.apache.org/jira/browse/KAFKA-6150
Kafka Streams deleting consumed r...
Rapprochement asked 30/1, 2021 at 18:59
2
Solved
We have a KStreams app in kotlin and switched from org.apache.kafka:kafka-streams:3.2.3 to org.apache.kafka:kafka-streams:3.3.1 and are now getting the following warning during runtime:
WARN org.ap...
Hopper asked 14/10, 2022 at 8:52
2
Solved
I have a Kafka Streams app that connects to our Kafka cluster using the Kafka Streams DSL, like so:
KStreamBuilder builder = new KStreamBuilder();
KStream<String, byte[]> stream = builder.st...
Whomever asked 11/1, 2017 at 22:20
2
Main question: we run Kafka Streams (Java) apps on Kubernetes to consume, process and produce real time data in our Kafka Cluster (running Confluent Community Edition v7.0/Kafka v3.0). How can we d...
Parsnip asked 22/2, 2022 at 13:51
1
I am using Kafka streams to read and process protobuf messages.
I am using the following properties for the stream:
Properties properties = new Properties();
properties.put(ConsumerConfig.GROUP_...
Lise asked 29/12, 2021 at 12:5
3
Solved
We are building a high-throughput low-latency stream processing application. We are using Apache Kafka both as a messaging platform and as a database.
It seems like the Kafka Streams and Alpakka K...
Cosmism asked 11/12, 2018 at 15:57
5
Solved
I am currently working with Akka Stream Kafka to interact with kafka and I was wonderings what were the differences with Kafka Streams.
I know that the Akka based approach implements the reactive ...
Maulmain asked 11/8, 2017 at 6:17
1 Next >
© 2022 - 2024 — McMap. All rights reserved.