Can I write to an AWS MSK Kafka cluster from a Lambda function?

Question

I need to pick up messages from AWS SES and publish them to an AWS MSK (Kafka) topic. Is this possible? My current understanding is that all MSK clients must live in the same VPC. How would I configure a Lambda function that appropriately?

Tartu

Answer 1

Short answer: Yes it is possible.

If your Source (Kafka Producer) and Target (Kafka Consumer) are in two different VPCs, then you can configure VPC Peering to access both Source and Target systems.

Napier

Answer 2

Inorder to connect to MSK cluster through lambda function, the lambda function needs to be in the same VPC of MSK.

But you can set up Rest proxy within MSK cluster and then perform rest call from lambda function (outside the VPC of MSK) to publich message into MSK cluster.

More informartion-

Amazon Managed Streaming for Kafka- MSK features and performance

Rhinology