Kryo serialization dependent on Java version?

Question

When deserializing a serialized object (from a file) using Kryo, I get the following exception:

java.lang.ExceptionInInitializerError
    (...)
Caused by: com.esotericsoftware.kryo.KryoException: (...)
Serialization trace: (...)
    at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
    at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
    at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:786)
    at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:143)
    at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:21)
    at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:682)
    (...)
Caused by: java.lang.IndexOutOfBoundsException: Index: 1582, Size: 2
    at java.util.ArrayList.rangeCheck(Unknown Source)
    at java.util.ArrayList.get(Unknown Source)
    at com.esotericsoftware.kryo.util.MapReferenceResolver.getReadObject(MapReferenceResolver.java:42)
    at com.esotericsoftware.kryo.Kryo.readReferenceOrNull(Kryo.java:830)
    at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:753)
    at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:113)
    ... 27 more

My hypothesis is that the serialized format is not properly understood when deserializing (ie. it changed). The Kryo version for serializing and deserializing was the same. The java version could have been different at the time of serialization: could this be an explanation?

If not, any other hints on what my generate such exceptions are more than welcome!

Many thanks, Thomas

UPDATE: as suggested, hereby the class that is being deserialized from the file

The main class deserialized is HashMap<Integer, PreflopEhsVO> where the custom class definitions (child and parent) are:

public class PreflopEhsVOExtended extends PreflopEhsVO{
    private int numbValues = 0;

    public synchronized void addValue(PreflopEhsVO values){
        if (numbValues == 0) this.valuesPerNumbOpp = values.valuesPerNumbOpp;
        else{
            //Weighted avg
            for (int i=0; i<this.valuesPerNumbOpp.length; ++i) this.valuesPerNumbOpp[i] = (this.valuesPerNumbOpp[i] * numbValues + values.valuesPerNumbOpp[i]) / (float) (numbValues + 1);
            ++numbValues;
        }
    }

    public PreflopEhsVOExtended(PreflopEhsVO values) {
        this.valuesPerNumbOpp = values.valuesPerNumbOpp;
        this.numbValues = 1;
    }
}

public class PreflopEhsVO {
    public float[] valuesPerNumbOpp = new float[9];

    public PreflopEhsVO(){
    }

    public PreflopEhsVO(float[] valuesPerNumbOpp) {
        this.valuesPerNumbOpp = valuesPerNumbOpp;
    }
}

Sector

Hard to see how. Can you post the class that was being deserialized? – Tuyere

Sure - just updated the post – Sector

I had the same problems with Kryo even without upgrading Java version. We've switched to FST - it's extremely stable comapred to Kryo – Myranda

What does your kryo instance look like? Which serializer did you use? Some more code would help. – Byrann

Answer 1

Kryo in general is not stable across different JVMs unless you are very careful about how you register types.

When Kryo encounters a type it hasn't seen before it increments a counter and registers the type to that value. It uses these values in the object graph as an alias for the type (an optimization for performance). If the execution order is slightly different on the second JVM (whether or not the version is the same), then Kryo will end up with different registries of aliased types. This prevents the serialized blobs from being deserialized.

The solution to this is to turn on "strict" mode in kryo and manually register the classes with explicit ids.

  kryo.register(SomeClass.class, 1);
  kryo.register(AnotherClass.class, 2);

This will ensure that the classes get registered to the same stable ids on both JVMs. If you do this, you should be able to get stable cross JVM serialization.

Another possible problem could be a change in the fields of the class pre and post serialization. The only practical way to handle changing fields is with the TaggedFieldSerializer

Loudmouthed