Comments in textual serialized protobuf? (not the scheme definition)

Question

I'm using textual protobuf files for system configuration.
One problem I have with this is that the serialized protobuf format does not support comments.
Is there any way around this?
I'm talking about the textual serialized data format, not the scheme definition.
Was this problem solved somewhere by someone?

Socioeconomic

I'm not a huge expert on the text variant (I mainly talk binary), hence not an answer: but AFAIK, simply "no". Are you sure it wouldn't be simpler to use JSON here? – Earnest

To use JSON I would need to use another third party library, (in C++) so I would rather not go there just yet. – Socioeconomic

JSON doesn't support comments. – Abrahan

Answer 1

Textual Protobuf format (serialized protobuf messages in text formal) supports comments using the # syntax. I could not find a reference for the same in any online documentation but have used the same in projects in the past so I put together a small example that one can test with:

Sample message description - [SampleProtoSchema.proto]

message SampleProtoSchema {
  optional int32 first_val = 1; // Note: This supports C/C++ style comments
  optional int32 second_val = 2;
}

Sample text message - [SampleTextualProto.prototxt]

# This is how textual protobuf format supports comments
first_val: 12 # can also be inline comments
# This is another comment
second_val: 23

Note though that these comments cannot be generated automatically at time of serialization. They can only be added manually afterwards.
Compile and test:

> protoc --python_out=. SampleProtoSchema.proto
>
> ipython
[1]: import SampleProtoSchema_pb2
[2]: sps = SampleProtoSchema_pb2.SampleProtoSchema()
[3]: from google.protobuf import text_format
[4]: with open('SampleTextualProto.prototxt', 'r') as f:
         text_format.Merge(f.read(), sps)
[5]: sps.first_val
[5]> 12
[6]: sps.second_val
[6]> 23

Scrivenor

As far as I can see that link describes the protobuf message description format, whereas the question is about the serialized text format. Do you have a link that supports the '//' theory? Otherwise I believe it is not supported. – Gearwheel

@MichaelSmith Sorry for the old answer. I must have misread. I fixed the answer and added an example since I did not find any reference online. – Scrivenor

Is there a way to have the message generator (i.e, when you create the message) to output the comments? Or do you have to output the message (configuration file) and then manually enter the comments. – Boaz

Answer 2

You may want to take a look at the Piqi project. It addresses this problem by introducing a new human-readable "Piq" data format and a command-line tool for converting data between Protobuf, Piq, JSON and XML formats.

The Piq data format was specially designed for human interaction. It supports comments, binary literals and verbatim text literals.

Abrahan