Can a C# model be serialized as an AVRO JSON Schema?

Question

I have found some code here https://learn.microsoft.com/en-us/azure/hdinsight/hdinsight-dotnet-avro-serialization#Scenario2 that does the reverse of what I need:

//Define the schema in JSON
const string Schema = @"{
    ""type"":""record"",
    ""name"":""Microsoft.Hadoop.Avro.Specifications.SensorData"",
    ""fields"":
        [
            {
                ""name"":""Location"",
                ""type"":
                    {
                        ""type"":""record"",
                        ""name"":""Microsoft.Hadoop.Avro.Specifications.Location"",
                        ""fields"":
                            [
                                { ""name"":""Floor"", ""type"":""int"" },
                                { ""name"":""Room"", ""type"":""int"" }
                            ]
                    }
            },
            { ""name"":""Value"", ""type"":""bytes"" }
        ]
}";

//Create a generic serializer based on the schema
var serializer = AvroSerializer.CreateGeneric(Schema);

I would like to take a model that I have created:

[DataContract(Name = "Demo", Namespace = "pubsub.demo")]
public class Demo
{
    [DataMember(Name = "value")]
    public long Value { get; set; }
}

...and serialize this C# model into a JSON AVRO Schema string.

Reason:

I only want to maintain C# models and automatically register these models with Confluent's Schema Registry. To register with schema registry the schema needs to be in a JSON AVRO format (Just like Schema above).

I would prefer not to have both the JSON defined and the C# model. If I had to maintian one, I would prefer to have a C# model.

Lampley

Late comment: The Confluent-dotnet-kafka library now includes avrogen, a tool to take a schema to a C# object – Noway

@cricket_007 is there any difference for the consumer if you publish message as ISpecificRecord or GenericRecord? Or those are still same byte arrays in message? – Howe

@Olegi The bytes will be the same – Noway

@cricket_007 thanks for the response! What is the prefered way to produce messages if your topic contains different event models? Should I produce specific records or generic records? Also, as a consumer, I think there is only one way to consume from that topic - is GenericRecord? Because basically you cannot pass an array of types you expect when you build your consumer. Really no samples for these scenarios on the web :( – Howe

@Howe You can watch this issue about support for multiple types in C# github.com/confluentinc/confluent-kafka-dotnet/issues/746 – Noway

Answer 1

I found what I was looking for in Microsoft.Hadoop.Avro.AvroSerializer.

AvroSerializer.Create<Demo>().WriterSchema.ToString();
// > {"type":"record","name":"pubsub.demo.Demo","fields"[{"name":"value","type":"long"}]}

Lampley

It doesn't seem like MS is supporting this library anymore github.com/Azure/azure-sdk-for-net/issues/… nor have the nuget packages been updated in several years nuget.org/packages?q=Microsoft.Hadoop.Avro – Bombshell

Answer 2

The solution also could be:

string schema = AvroConvert.GenerateSchema(typeof(Demo));

From https://github.com/AdrianStrugala/AvroConvert

Arie

Answer 3

Chr.Avro provides a CLI that lets you go both ways- C# types to Avro schema, and Avro schema to C# types.

From the docs:

dotnet avro create --type ExampleNamespace.ExampleLibrary.ExampleClass --assembly bin/Debug/netstandard2.0/ExampleNamespace.ExampleLibrary.dll

https://engineering.chrobinson.com/dotnet-avro/guides/cli-create

Sawtelle