I'm working on developing an .NET Standard pub/sub package using Kafka and Confluent's .NET client. My producer has the following interface. IEventPublisher.cs public interface IEventPublisher&l...

I'm using Kafka connect HDFS. When I'm trying to run my connector I'm got the following exception: ERROR Failed creating a WAL Writer: Failed to create file[/path/log] for [DFSClient_NONMAPREDUC...

I am using kafka stream download from Confluent (http://www.confluent.io/product/kafka-streams/). I am following the instructions to run Zookeeper and Kafka on Windows. But while I try to start Z...

I want to read only 5000 records in a batch through jdbc sink, for which I've used the batch.size in the jdbc sink config file: name=jdbc-sink connector.class=io.confluent.connect.jdbc.JdbcSinkCon...

When the consumer does not receives a message for 5 mins (default value of max.poll.interval.ms 300000ms) the consumer comes to a halt without exiting the program. The consumer process hangs and do...

I have implemented a Kafka consumer as a console app by using BackgroundService on .NET Core 2.2. I am using confluent-kafka-dotnet v1.0.1.1 as a client for Apache Kafka. I'm not so sure about how ...

I'm using kafka-avro-console-producer from confluent-3.0.0 and error occurs when I execute the following: ./bin/kafka-avro-console-producer --broker-list localhost:9092 --topic test1234 --property...

I am trying to configure ssl to confluent kafka docker platform and getting an error while starting saying Logs: Command [/usr/local/bin/dub path /etc/kafka/secrets/kafka.server.keystore.jks exis...

My local win10 WSL back unbuntu, In ubuntu, i have installed spark3.0.0, confluent platform 5.5.1 (manually downloaded) when I try to run either spark-shell or spark-submit (below is shell example)...

OS: Ubuntu 18.x docker image (from dockerhub.com, as of 2020-09-25): confluentinc/cp-schema-registry:latest I am exploring the HTTP API for the Confluent Schema Registry. First off, is there a def...

I'm using Kafka & Schema-registry. Defined a schema, using confluent's KafkaAvroSerializer on the producer side. Everything works fine. On the other hand, if a producer publishes the event wit...

I set up Kafka for local run. I have written sample producer and consumer in Java and running from local, by starting server and zookeeper. I want to use oracle as producer, that will require to wr...

is there any Python kafka admin client avilable to create topic/delete topic from python program ? I found some python apis but none of them have Admin api available ? Does confluent have python ...

Hi I'm currently setting up Kafka with Docker. I've managed to setup Zookeeper and Kafka with the published confluent image, see following docker-compose file: version: '2' services: zookeeper: ...

I am using Kafka Connect from Confluent to consume Kafka stream and write to HDFS in parquet format. I am using Schema Registry service in 1 node and it is running fine. Now I want to distribute Sc...

I am going through the documentation looking at multiple places, it is adding up confusion.. About the property min.insync.replicas When a producer sets acks to "all" (or "-1"), this configura...

Occasionally with Kafka Connect, I see my JdbcSourceConnector's task go up and down--that is, the REST interface sometimes reports one task that is RUNNING and sometimes reports no tasks (the conne...

I have installed Kafka on a local Minikube by using the Helm charts https://github.com/confluentinc/cp-helm-charts following these instructions https://docs.confluent.io/current/installation/instal...

I know , in KSQL we can set offset to earliest or latest But can we get data from specific time period i.e I need to get data inserted to a topic from 06-May-2020 ?

To develop my Kafka connector I need to add a connect-API dependency. Which one I should use? For example mongodb connector use connect-api from maven central But links from dev guide go to http...

Instruction page says as:- Install the Confluent Cloud CLI Run this command to install the Confluent Cloud CLI. curl -L --http1.1 https://cnfl.io/ccloud-cli | sh -s -- -b /usr/local/bin it fail...

Im trying to setup Kafka Connect with the intent of running a ElasticsearchSinkConnector. The Kafka-setup, consisting of 3 brokers secured using Kerberos, SSL and and ACL. So far Ive been experim...

Suppose I have different values for a same message key. For example: { userid: 1, email: [email protected] } { userid: 1, email: [email protected] } { userid: 1, email: [email p...

I am using confluent-kafka-python and see that it hangs infinitely when I try to connect to a broker which is down. I can't seem to apply any time out settings I found in the docs: from confluent_...

The Consumer instance from confluent-kafka python client always returns None when calling poll() with timeout set. The topic does contains some message and the official console consumer works fin...