I successfully installed Postgres Debezium CDC. Now, I'm able to catch all changes happening to the database. But the problem is "before" field always stays empty. So, if I insert a record (id = 1,...

I'm trying to sink the table data one DB to another DB using kafka debezium ( Kafka streaming ) with the help of docker. DB stream is working fine. But streamed data to sink another MySQL DB proce...

After starting Kafka Connect (connect-standalone), my task fails immediately after starting with: java.lang.OutOfMemoryError: Java heap space at java.nio.HeapByteBuffer.<init>(HeapByteBuffe...

On port 8083 I am running Influxdb for which I am even getting the GUI on http://localhost:8083 Now come to kafka, Here I am referring the setup as per https://kafka.apache.org/quickstart starti...

I'm working a couple of Kafka connectors and I don't see any errors in their creation/deployment in the console output, however I am not getting the result that I'm looking for (no results whatsoev...

I am using native apache kafka and not confluent's edition of kafka. I am using using kafka jdbc connector pluggin. I intend to put a break point and inspect through my IDE (intelliJ). How to mak...

I have set up Kafka cluster and installed confluent-hub on a EC2 instance. I have downloaded the confluent-hub tar file, extracted and placed the binary inside /usr/local/bin. When I am now trying ...

I am running a data pipeline such that I read data from sql db into kafka topic through jdbc connect I sink this data in Elasticsearch using kafka sink connector for ES I have a need to reset thi...

I have an already running production deployed Kafka-Cluster and having Topic "existing-topic". I am using MongoDB-Source-Connector from Debezium. Here all what I want is to push the CDC events di...

I have been trying this for two weeks now, I am running Kafka cluster on separate machines than my connect nodes. I am unable to get connect running properly. I can read and write to kafka no issue...

I have searched quite a lot on this but there doesn't seems to be a good guide around this. From what I have searched there are a few things to consider: Resetting Sink Connector internal topics...

I am using the confluent docker for Kafka connect v4.1.1 and would like to add a jar with specific log4j appenders. With the classpath isolation by connector, I am not sure where to place that jar...

I have a 3-nodes kafka-connect worker cluster in distributed mode, with a running s3 sink connector. To update the configuration of the connector at run-time, I run the command below: curl -X PUT...

I've just deployed my Kafka Connect (I only use a connect source to MQTT) application on a cluster of two instances (2 containers on 2 machines) and now it seems to get into a sort of rebalancing l...

This question is related to: Debezium How do I correctly register the SqlServer connector with Kafka Connect - connection refused In Windows 10, I have Debezium running on an instance of Microsoft...

We are running Kafka Connect (Confluent Platform 5.4, ie. Kafka 2.4) in a distributed mode using Debezium (MongoDB) and Confluent S3 connectors. When adding a new connector via the REST API the con...

Consider the Kafka Producer : const { Kafka, logLevel } = require('kafkajs'); const kafka = new Kafka({ clientId: 'my-app', brokers: ['kafka1:9092', 'kafka2:9092'], logLevel: logLevel.ERROR, })...

I had a working s3 sink connector until the source connector sent a NULL value; s3 connector crashed. The problem occured when I deleted a record from MS SQL db. The source connector shipped the de...

kafka connect 5.4 only one connector, one worker and use connect-distributed. Below is the error message: [2020-06-22 19:09:58,700] ERROR [Worker clientId=connect-1, groupId=test-cluster] Uncaught...

I have topics in Kafka that are stored in Avro format. I would like to consume the entire topic (which at time of receipt will not change any messages) and convert it into Parquet, saving directly ...

We build a custom Kafka Connect sink which in turn calls a remote REST API. How do I propagate backpressure to the Kafka Connect infrastructure, so put() is called less often in cases when th...

I installed kafka confluent oss 4.0 on a fresh linux centos 7 but kafka connect failed to start. Steps to reproduce : - Install Oracle JDK 8 - Copy confluent-4.0.0 folder on opt/confluent-4.0.0...

I tried to deploy ElasticsearchSinkConnector by POST /connectors { "name": "elasticsearch-sink", "config": { "connector.class": "io.confluent.connect...

What are the differences between JDBC Connector and Debezium SQL Server CDC Connector (or any other relational database connector) and when should I choose one over another, searching for a solutio...

I'm trying to use the Kafka Connect Elasticsearch connector, and am unsuccessful. It is crashing with the following error: [2018-11-21 14:48:29,096] ERROR Stopping after connector error (org.apach...