I have a no-sql db where I am storing serialized avro objects. I use sbt-avro plugin to generate java classes specifically addSbtPlugin("com.cavorite" % "sbt-avro-1-8" % "1.1.5"). Now I want to add new fields to the schema and also want backward compatibility with the data already present. But deserialization of old data using new schema is throwing java.io.EOFException
Old Schema:
{
"name": "DNode",
"namespace": "my.model",
"type": "record",
"fields": [
{"name": "prev", "type": ["null", "string"]},
{"name": "next", "type": ["null", "string"]}
]
}
New Schema:
{
"name": "DNode",
"namespace": "my.model",
"type": "record",
"fields": [
{"name": "prev", "type": ["null", "string"]},
{"name": "next", "type": ["null", "string"]},
{"name": "size", "type": ["null", "long"], "default": null}
]
}
Deserialization code
def deserialize(bytes: Array[Byte]) = {
val reader = new SpecificDatumReader[DNode](classOf[DNode])
val decoder = DecoderFactory.get().binaryDecoder(bytes, null)
reader.read(null, decoder)
}
Throws
java.io.EOFException
at org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:473)
at org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:128)
at org.apache.avro.io.BinaryDecoder.readIndex(BinaryDecoder.java:423)
at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:290)
at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:267)
at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:179)
at org.apache.avro.specific.SpecificDatumReader.readField(SpecificDatumReader.java:116)
at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:222)
at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:145)
...