Java8 Lambda Deserialization - ClassCastException

Question

ClassCastException is thrown by Java8 upon deserializing a lambda when following conditions are met:

• Parent class has a method, reference to which is used to automatically create a Serializable lambda
• There are several child classes that extend it and there are several usages of above method as a method reference, but with different child classes
• After method reference is consumed it is serialized and the deserialized
• All method references are used within the same capturing class

Tested on Oracle Java compiler and runtime versions 1.8.0_91. Please find test code on how to reproduce:

import java.io.*;

/**
 * @author Max Myslyvtsev
 * @since 7/6/16
 */
public class LambdaSerializationTest implements Serializable {

    static abstract class AbstractConverter implements Serializable {
        String convert(String input) {
            return doConvert(input);
        }

        abstract String doConvert(String input);
    }

    static class ConverterA extends AbstractConverter {
        @Override
        String doConvert(String input) {
            return input + "_A";
        }
    }

    static class ConverterB extends AbstractConverter {
        @Override
        String doConvert(String input) {
            return input + "_B";
        }
    }

    static class ConverterC extends AbstractConverter {
        @Override
        String doConvert(String input) {
            return input + "_C";
        }
    }

    interface MyFunction<T, R> extends Serializable {
        R call(T var);
    }

    public static void main(String[] args) throws Exception {
        System.out.println(System.getProperty("java.version"));
        ConverterA converterA = new ConverterA();
        ConverterB converterB = new ConverterB();
        ConverterC converterC = new ConverterC();
        giveFunction(converterA::convert);
        giveFunction(converterB::convert);
        giveFunction(converterC::convert);
    }

    private static void giveFunction(MyFunction<String, String> f) {
        f = serializeDeserialize(f);
        System.out.println(f.call("test"));
    }

    private static <T> T serializeDeserialize(T object) {
        try {
            ByteArrayOutputStream baos = new ByteArrayOutputStream();
            ObjectOutputStream oos = new ObjectOutputStream(baos);
            oos.writeObject(object);
            byte[] bytes = baos.toByteArray();
            ByteArrayInputStream bais = new ByteArrayInputStream(bytes);
            ObjectInputStream ois = new ObjectInputStream(bais);
            @SuppressWarnings("unchecked")
            T result = (T) ois.readObject();
            return result;
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

}

It gives following output:

1.8.0_91
test_A
Exception in thread "main" java.lang.RuntimeException: java.io.IOException: unexpected exception type
    at LambdaSerializationTest.serializeDeserialize(LambdaSerializationTest.java:68)
    at LambdaSerializationTest.giveFunction(LambdaSerializationTest.java:52)
    at LambdaSerializationTest.main(LambdaSerializationTest.java:47)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: java.io.IOException: unexpected exception type
    at java.io.ObjectStreamClass.throwMiscException(ObjectStreamClass.java:1582)
    at java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1154)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1817)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
    at LambdaSerializationTest.serializeDeserialize(LambdaSerializationTest.java:65)
    ... 7 more
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at java.lang.invoke.SerializedLambda.readResolve(SerializedLambda.java:230)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1148)
    ... 11 more
Caused by: java.lang.ClassCastException: LambdaSerializationTest$ConverterB cannot be cast to LambdaSerializationTest$ConverterA
    at LambdaSerializationTest.$deserializeLambda$(LambdaSerializationTest.java:7)
    ... 21 more

Upon decompiling this $deserializeLambda$ method with CFR following code is revealed:

private static /* synthetic */ Object $deserializeLambda$(SerializedLambda lambda) {
    switch (lambda.getImplMethodName()) {
        case "convert": {
            if (lambda.getImplMethodKind() == 5 && lambda.getFunctionalInterfaceClass().equals("LambdaSerializationTest$MyFunction") && lambda.getFunctionalInterfaceMethodName().equals("call") && lambda.getFunctionalInterfaceMethodSignature().equals("(Ljava/lang/Object;)Ljava/lang/Object;") && lambda.getImplClass().equals("LambdaSerializationTest$AbstractConverter") && lambda.getImplMethodSignature().equals("(Ljava/lang/String;)Ljava/lang/String;")) {
                return (MyFunction<String, String>)LambdaMetafactory.altMetafactory(null, null, null, (Ljava/lang/Object;)Ljava/lang/Object;, convert(java.lang.String ), (Ljava/lang/String;)Ljava/lang/String;)((ConverterA)((ConverterA)lambda.getCapturedArg(0)));
            }
            if (lambda.getImplMethodKind() == 5 && lambda.getFunctionalInterfaceClass().equals("LambdaSerializationTest$MyFunction") && lambda.getFunctionalInterfaceMethodName().equals("call") && lambda.getFunctionalInterfaceMethodSignature().equals("(Ljava/lang/Object;)Ljava/lang/Object;") && lambda.getImplClass().equals("LambdaSerializationTest$AbstractConverter") && lambda.getImplMethodSignature().equals("(Ljava/lang/String;)Ljava/lang/String;")) {
                return (MyFunction<String, String>)LambdaMetafactory.altMetafactory(null, null, null, (Ljava/lang/Object;)Ljava/lang/Object;, convert(java.lang.String ), (Ljava/lang/String;)Ljava/lang/String;)((ConverterB)((ConverterB)lambda.getCapturedArg(0)));
            }
            if (lambda.getImplMethodKind() != 5 || !lambda.getFunctionalInterfaceClass().equals("LambdaSerializationTest$MyFunction") || !lambda.getFunctionalInterfaceMethodName().equals("call") || !lambda.getFunctionalInterfaceMethodSignature().equals("(Ljava/lang/Object;)Ljava/lang/Object;") || !lambda.getImplClass().equals("LambdaSerializationTest$AbstractConverter") || !lambda.getImplMethodSignature().equals("(Ljava/lang/String;)Ljava/lang/String;")) break;
            return (MyFunction<String, String>)LambdaMetafactory.altMetafactory(null, null, null, (Ljava/lang/Object;)Ljava/lang/Object;, convert(java.lang.String ), (Ljava/lang/String;)Ljava/lang/String;)((ConverterC)((ConverterC)lambda.getCapturedArg(0)));
        }
    }
    throw new IllegalArgumentException("Invalid lambda deserialization");
}

So it appears that actual captured argument is not used to determine which exact lambda has to be deserialized. All 3 lambdas will satisfy 1st if condition and ConverterA will be assumed.

When debugging we can observe that in runtime lambda.getCapturedArg(0) is of a correct type (ConverterB when exception is thrown) and also it worth noting that cast is not needed since method to be invoked is present in base AbstractConverter class.

Is it expected behavior? If yes, what is recommended workaround?

Hypo

On my PC it works like a charm with output:1.8.0_60 test_A test_B test_C – Zeebrugge

Could it be platform specific? I am using Mac OS. – Hypo

I am also using Mac :) – Zeebrugge

Could you please try decompiling your $deserializeLambda$ method with CFR like this: java -jar cfr_0_115.jar LambdaSerializationTest.class --removeboilerplate false --decodelambdas false – Hypo

If you suspect a bug, you should always include the exact compiler version, i.e. which jdk and/or whether you use a custom compiler like Eclipse. – Ninurta

Of course, this is a bug. I don’t think that the $deserializeLambda$ method should test the types of captured values, but the compiler should not encode the fact that the method convert is inherited from AbstractConverter. Instead, it should encode exactly what you have written, i.e. references to ConverterA::convert, ConverterB::convert, and ConverterC::convert, just like it would for ordinary method invocations. If it did, the implClass comparison would differentiate the creation sites. – Ninurta

@Alexander Petrov: are you using Eclipse or any other IDE with its own compiler? – Ninurta

Yes I compiled it in Eclipse. But it is not using Own compiler it has the same compiler set that is in the command line. – Zeebrugge

@Alexander Petrov: Eclipse has its own compiler. I even don’t know of any option to tell Eclipse to use javac instead of its builtin compiler (besides using an external build tool like ant). – Ninurta

OK i will try from the console in an hour and let you know how IT goes. – Zeebrugge

@Holger: I would not expect the compiler to reference exact ConverterX::convert methods, because those methods do not actually exist (there is only AbstractConverter::convert method). So instead I would prefer the compiler to generate just a single if condition and cast to AbstractConverter inside of it. – Hypo

These methods do exist, the fact that these ConverterX classes inherit the method rather than declaring or overriding it, is an implementation detail of the ConverterX classes which should not affect the code of the caller. Think about it: why should the decision of overriding the method in ConverterX (or not) change the code of the caller? That would imply that you have to recompile your applications for every new JRE release… – Ninurta

@Holger: agreed, I was just confused by the fact that ConverterX does not declare convert method. But if you just invoke ConverterA.convert() directly, following bytecode will be generated INVOKEVIRTUAL LambdaSerializationTest$ConverterA.convert implying that ConverterA has a virtual method convert which proves your point. – Hypo

So is there a resolution ? – Zeebrugge

I have sent a bug report to Oracle and they are currently evaluating it. I will post here in case of any updates. – Hypo

That’s why I wrote “just like it would for ordinary method invocations”. I remember times, when Java 1.2 was really new, where the compiler did use the declaring class in invocations, leading to surprising behavior, e.g. a Java 1.1 compatible program, recompiled under 1.2 with -target 1.1 still failed to run under 1.1 due to references to 1.2 specific classes it never referenced by itself. But that was almost twenty years ago… – Ninurta

Oracle has confirmed a bug, updated the post with details. – Hypo

@Max Myslyvtsev: I turned your last update into an answer as I think, that’s more appropriate. – Ninurta

Answer 1

Oracle has confirmed that it is a bug and assigned a following Bug ID: JDK-8161257

It is now visible on the official tracker: JDK-8161257

Ninurta