Quick question for those who have used Amazon's MSK (Managed Streaming for Kafka): The docs/FAQ make it clear that you can easily scale up the number of brokers in an MSK cluster. Can you just as ...

I've written a python script with aiokafka to produce and consume from a Kafka cluster in AWS MSK, I'm running the script from a EC2 instance that is in the same VPC as my cluster and when I try to...

I am trying to connect to AWS MSK using SASL protocol in a spring boot project. I am using spring-kafka to connect. The error I am getting is org.apache.kafka.common.KafkaException: javax.security...

we have an AWS cloudformation template, through which we are creating the Amazon MSK(Kafka) cluster. which is working fine. Now we have multiple applications in our product stack which consume the...

We're trying to use AWS MSK (managed Kafka), and we want to use AWS GLUE Schema registry with AVRO rather than Confluent Schema Registry. We have brought up KSQLDB, and connected that to MSK, but i...

I am trying to use MongoDB change stream using Kafka. I chose AWS MSK since my entire infrastructure is within AWS and it will be easy to integrate with other AWS services. I created an AWS MSK clu...

I am deciding if I should use MSK (managed kafka from AWS) or a combination of SQS + SNS to achieve a pub sub model? Background Currently, we have a micro service architecture but we don't use any ...

I have AWS MSK Kafka cluster with 2 brokers. From the logs I can see (on each broker) that they are constantly rebalancing. Every minute I can see in logs: Preparing to rebalance group amazon.msk.c...

We are planning to use AWS MSK service for Managed Kafka and Schema Registry and Kafka Connect services from Confluent together to run our connectors (Elasticsearch Sink Connector). We have planned...

I'm trying to figure out how to create a working Kafka producer against an AWS MSK cluster. The cluster has TLS and SASL/SCRAM enabled. This is my first time using Kafka and the documentation on th...

I am trying to create an Kafka client app (both producer and consumer) using AWS managed Kafka instance (MSK). Also broker to broker communication and client to broker communication is configured a...

I am having trouble connecting to AWS MSK Kafka (with with Access Control Method configured to None) from my laptop, error is: NoBrokersAvailable I am using plaintext conn over port 9092: from kafk...

Does anyone have any idea if AWS MSK (Managed Streaming for Kafka) supports KSQL? I have seen many videos and documentations for KSQL on confluent platform but not for AWS MSK. Please let me know i...

Is there any info as to how can I use the AWS MSK details in my Spring Cloud Stream application ? I believe we need to generate a keystore and truststore and then incorporate the same in our appli...

I'm trying to connect my python consumer to AWS MSK cluster. how can I do that? Have an AWS MSK Cluster running I'm trying consume message from the MSK cluster using python and kafka python. ...

I am getting this error when constructing KafkaProducer with the kafka-python package: [ERROR] UnrecognizedBrokerVersion: UnrecognizedBrokerVersion Traceback (most recent call last): File "/var/l...

Recently we have started doing some testing of our Kafka connectors to MSK, Amazon's managed Kafka service. Publishing records seem to work fine however not when transactions are enabled. Our clust...

I'm evaluating AWS Kinesis vs Managed Service Kafka (MSK). Our requirement is sending some messages (JSON) to AWS to from the on-prem system (system develop using c++). Then we need to persist abov...

I'm using AWS MSK and I want to enable ACLs but I'm unable to create a topic when ACLs are turned-on. I'm using the command-line tools for all the operations. Here's a summary of what I'm doing: ...

Our AWS MSK cluster is not automatically creating topics when a producer first publishes a message to the topic. This is surprising because the default behaviour according to https://kafka.apache.o...

I need to pick up messages from AWS SES and publish them to an AWS MSK (Kafka) topic. Is this possible? My current understanding is that all MSK clients must live in the same VPC. How would I confi...

I'm evaluating AWS Managed Service Kafka (MSK) and I know that currently, it is in preview mode, hence might not have all features or proper documentation. I tried setting up the msk cluster and wa...