How do I generate a .proto file from a C# class decorated with attributes?

Question

Trying to get my mind around google protobuf. I found some implementation of protobuf in C# but they seems to lack one feature: the ability to generate .proto files automatically from an existing C# class decorated with attributes.

The reason I want to do it this way instead of going from auto-generated C# classes from .proto file is because I already have the C# classes defined in my project and I don't want to duplicate them just to satisfy ProtoBuf.

Does anyone have encountered such a scenario?

---

Update

Is this possible to just decorate a C# class and not use a .proto file to use protobuf?

Undertake

Answer 1

Good news; what you have described (having existing C# classes) is the expected use-case of protobuf-net. All the .proto stuff ("protogen", the VS add-in, etc) were all added as afterthoughts. The core of protobuf-net doesn't know about them or care about them.

protocol buffers defines a DSL (.proto, as you mention) that is shared between implementations, and is (sometimes) used for code generation. When I first wrote protobuf-net, the code-generation aspect wasn't my biggest concern - simply that .NET developers are generally guilty (myself included) of "implementation first" rather than "contract first".

As a consequence, protobuf-net doesn't need .proto files to work; an attributed class is sufficient to unambiguously serialize/deserialize. Just use Serializer.Serialize , .Merge and .Deserialize (etc).

That said; it does include some very under-developed and experimental support for this:

string proto = Serializer.GetProto<YourType>();

This is far from complete, but may work for simple types. If you have some specific cases where it fails, then let me know (add a comment or log an issue). However; most of the time, people interested in .proto would write the .proto first and work from there.

Examples of working decorated types are shown on the project home page; it is entirely up to you whether you use WCF attributes, xml attributes or protobuf-net attributes (although the latter provide more control over some specific serialization points, such as inheritance and numeric layouts).

Marijo

Great, this would fit my requirements then. However, one more thing, does Protobuf-Net implements the whole specification of Protobuf? – Grider

Pretty-much; the wire-format spec isn't vast, to be honest. Are you looking to interop with a separate .proto implementation? It should work fine, but if you have a .proto (for the other end), I'd recommend generating a DTO (from the .proto) and shim to that from your types. It also gets creative, allowing protobuf-net to support inheritance (which is not part of the core .proto spec), but done in a way that still allows full interop with other clients. – Marijo

I am not looking to interop with another implementation. I will use protobuf-net as an efficient mechanism for serializing/deserializing a whole hierarchy of objects between two applications. – Grider

One other thing, is it possible to have a serialization of a property defined as an IList but the actual object is a List? – Grider

That should be fine as long as your class creates the concrete list. If it doesn't work, let me know (I'll try to remember to test it on the train tomorrow). – Marijo

I've just checked, and a pre-initialized IList<T> works fine; I've also tweaked it to use List<T> if the list isn't pre-initialized. – Marijo

You're right! I was not initializing the concrete list in my private constructor. I like ProtoBuf-Net more and more... :) – Grider

@Marc Gravell - How does this fair with generating .proto definitions for protobuf "services"? We're adding protobuf-net support to an existing Web Service so I'd like to keep the service as "the definition" rather than create a .proto from scratch. The client is not .NET so we can't go .proto-less – Schoof

@Richard - nothing written there as-yet, but you could probably hack something together via reflection. – Marijo

Is this still possible? I have tried this and looked at the current version (r480) and trunk and they seem to throw NotImplementedException. – Urbani

@Mike it will be, but not today - it is perhaps the largest thing not yet reimplemented – Marijo

Marc thanks for your great support on .NET and Protobuf but as you said "protobuf-net doesn't need .proto files to work; an attributed class is sufficient to unambiguously serialize/deserialize" it kind of breaks the rules. There is no another api in other platforms which does not need .proto files. And Google's definition also requires it, so protobuf-net is cool and easy to use with attributes, it kind of breaks interoperability and point to use protobuf. Its name should be corrected as "protobuf based serializer for .net" since its not the protobuf as creators intented. – Saber

@Saber my intent is not to tick boxes with Google; my intent is to provide a tool that is useful, convenient, and compatible; if you want a .proto file, there is functionality built in to export one, as shown in the answer. – Marijo

@MarcGravell What's the current state now (5 years later)? Does .proto generation still work, and does it also work for gRPC? Our use case is that we have existing C# interfaces which we want to gRPCify, but also open them for clients in other languages. – Cene

@Cene yes, protobuf-net.Grpc.Reflection - something like SchemaGenerator IIRC - it isn't as complete as the schema generation from protobuf-net, though – Marijo

I am so confused with protobuf-net. When I add a .proto file to the project, VS generates a file that I can use to generate my service endpoints and my client. I don't have to do another thing to consume that client. How does protobuf-net work here? Is it ONLY for generating the models? Do I still need to manually create a client project that consumes these models? How do I implement the service? Do I have to manually serialize/deserialize the models? Where is the documentation for this thing?!? – Gradygrae

@SerjSagan it sounds like you're talking about services, so: more the gRPC side of things than just serialization, in which case: protobuf-net.github.io/protobuf-net.Grpc (the "getting started" page is a good place to ... start) - and since you seem to be working "schema first" (i.e. from a .proto file), the build tools are documented here: protobuf-net.github.io/protobuf-net/contract_first - however! I can't see your csproj, but I wonder if you're actually using the Google build tools, not the protobuf-net ones – Marijo

Answer 2

Before Skeet Marc runs in here and gets massive ups, let me point out protobuf.net.

Sitarski

Actually, that would be me ;-p – Marijo

Yeah, I looked at the project but can't seem to find documentation about how to generate .proto files from a decorated C# class. Or is it needed to have a .proto file to use protobuf at all? – Grider

Oh, Jon has a pb implementation; but a different one; dotnet-protobufs: github.com/jskeet/dotnet-protobufs/tree/master – Marijo

And for the record, it wouldn't be a "fit" for this question (else I would have happily recommended it) - as it uses the codegen approach exclusively (exactly what the OP doesn't want). – Marijo