Using AWS glue schema registry with confluent SerDe clients

Question

For supporting schema registry on my MSK topic, I found two options -

• AWS Glue Schema Registry; and
• Confluent Schema Registry

Since, Glue SR is fully managed by AWS, I would prefer to use that. However, my producer and consumer clients are written in python, which limits me from using SerDe libraries provided by AWS in java.

I went on to search whether the confluent schema registry API (part of confluent kafka library in python) can be made to work with the Glue Schema Registry, since my initial assumption is that the schema registry implementations are generic across confluent and glue.

When reading confluent documentation, I found that a schema registry connection can be established by supplying url for schema registry using url property (https://docs.confluent.io/platform/current/clients/confluent-kafka-python/#).

I cannot find such url for the glue registry I created. Their java client does take in properties like region name, registry name and other properties (https://docs.aws.amazon.com/glue/latest/dg/glue-dg.pdf#schema-registry). Is there any way this url can be found out, or should I switch to confluent SR, which is not a managed service?

Cesaro

I am facing a similar issue with my Node.js and Ruby on Rails project. It would be great if they release the support soon. – Savaii

did you find any workaround for this? – Savaii

I have the same problem with dotnet code and aws glue, did you find any solution for that ? – Preordain

Answer 1

AWS Glue Schema Registry does not have the notion of a URL endpoint. You interact with the service through the AWS APIs given a region and a registry name.

AWS has provided a library

https://github.com/awslabs/aws-glue-schema-registry

Slipover

Are you aware if there are aws-glue-schema-registry libraries in other languages, but Java? For example, dotnet. – Gradate

Did you manage to find something on this.? Like for eg a python version for using in pyspark.? – Inexcusable