Can you put comments in Avro JSON schema files?

Question

I'm writing my first Avro schema, which uses JSON as the schema language. I know you cannot put comments into plain JSON, but I'm wondering if the Avro tool allows comments. E.g. Perhaps it strips them (like a preprocessor) before parsing the JSON.

Edit: I'm using the C++ Avro toolchain

Paraldehyde

Answer 1

Yes, but it is limited. In the schema, Avro data types 'record', 'enum', and 'fixed' allow for a 'doc' field that contains an arbitrary documentation string. For example:

{"type": "record", "name": "test.Weather",
 "doc": "A weather reading.",
 "fields": [
     {"name": "station", "type": "string", "order": "ignore"},
     {"name": "time", "type": "long"},
     {"name": "temp", "type": "int"}
 ]
}

From the official Avro spec:

doc: a JSON string providing documentation to the user of this schema (optional).

https://avro.apache.org/docs/current/spec.html#schema_record

An example: https://github.com/apache/avro/blob/33d495840c896b693b7f37b5ec786ac1acacd3b4/share/test/schemas/weather.avsc#L2

Contrabassoon

Answer 2

Yes, you can use C comments in an Avro JSON schema : /* something */ or // something
Avro tools ignores these expressions during the parsing.
EDIT: It only works with the Java API.

Avellaneda

This doesn't work for me. If I put either style of comments (/**/ or //) into the JSON file, I get an error: "Failed to parse or compile schema: Unexpected character in json 2f". I tried putting both style of comments in several places in the file. – Paraldehyde

I suggest that you have ever compiled your schema without the comments and it works. Check the following things : latest version of avro-tools, keep sure that your schema compiles (with the avro tools or Json validator). For your information, comments work fine for all my avro schema... – Avellaneda

I'm using 1.7.4 which is the latest release. I think the issue might be that we're using the C++ version right now, and you're using the Java version. Sorry to have left out that detail in my question. – Paraldehyde

Indeed... I just tried with the C++ version and it doesn't work. Thanks to highlight this avro issue. – Avellaneda

Same with python package Avro. It internally uses the straight json loader to parse the schema and it fails if there are comments or any other "extended" json syntax. – Refutation

Answer 3

According to the current (1.9.2) Avro specification it's allowed to put in extra attributes, that are not defined, as metadata:

This allows you add comments like this:

{
  "type": "record", 
  "name": "test",
  "comment": "This is a comment",
  "//": "This is also a comment",
  "TODO": "As per this comment we should remember to fix this schema" ,
  "fields" : [
    {
      "name": "a", "type": "long"
    },
    {
      "name": "b", "type": "string"
    }
  ]
}

Dacha

Answer 4

No, it can't in the C++ nor the C# version (as of 1.7.5). If you look at the code they just shove the JSON into the JSON parser without any comment preprocessing - bizarre programming style. Documentation and language support appears to be pretty sloppy...

Zloty