Why does the Java 8 generic type inference pick this overload?

Question

Consider the following program:

public class GenericTypeInference {

    public static void main(String[] args) {
        print(new SillyGenericWrapper().get());
    }

    private static void print(Object object) {
        System.out.println("Object");
    }

    private static void print(String string) {
        System.out.println("String");
    }

    public static class SillyGenericWrapper {
        public <T> T get() {
            return null;
        }
    }
}

It prints "String" under Java 8 and "Object" under Java 7.

I would have expected this to be an ambiguity in Java 8, because both overloaded methods match. Why does the compiler pick print(String) after JEP 101?

Justified or not, this breaks backward compatibility and the change cannot be detected at compile time. The code just sneakily behaves differently after upgrading to Java 8.

NOTE: The SillyGenericWrapper is named "silly" for a reason. I'm trying to understand why the compiler behaves the way it does, don't tell me that the silly wrapper is a bad design in the first place.

UPDATE: I've also tried to compile and run the example under Java 8 but using a Java 7 language level. The behavior was consistent with Java 7. That was expected, but I still felt the need to verify.

Faught

I tried with eclipse with jdk 1.8.0 or jdk1.8.45, it prints Object – Hailstorm

I got String on Ideone.com (which uses sun-jdk-8u25). – Trudey

I would change override to overload in the question title and body. There's no overriding involved. – Trudey

@Giovanni: I suppose we should ban any Eclipse-specific statement from Java 8 related questions for the next years… (don’t get me wrong, I really wish to use Eclipse with Java 8) – Jacy

I got String on Oracle jdk1.8.0_45. – Woodsman

This is really a bad change in Java 8. It just bit me hard. I was using methods that return an unknown type similar to the SillyGenericWrapper, now that probably doesn't work anymore as wrong types will be inferred all the time. – Smite

Answer 1

The rules of type inference have received a significant overhaul in Java 8; most notably target type inference has been much improved. So, whereas before Java 8 the method argument site did not receive any inference, defaulting to Object, in Java 8 the most specific applicable type is inferred, in this case String. The JLS for Java 8 introduced a new chapter Chapter 18. Type Inference that's missing in JLS for Java 7.

Earlier versions of JDK 1.8 (up until 1.8.0_25) had a bug related to overloaded methods resolution when the compiler successfully compiled code which according to JLS should have produced ambiguity error Why is this method overloading ambiguous? As Marco13 points out in the comments

This part of the JLS is probably the most complicated one

which explains the bugs in earlier versions of JDK 1.8 and also the compatibility issue that you see.

---

As shown in the example from the Java Tutoral (Type Inference)

Consider the following method:

void processStringList(List<String> stringList) {
    // process stringList
}

Suppose you want to invoke the method processStringList with an empty list. In Java SE 7, the following statement does not compile:

processStringList(Collections.emptyList());

The Java SE 7 compiler generates an error message similar to the following:

List<Object> cannot be converted to List<String>

The compiler requires a value for the type argument T so it starts with the value Object. Consequently, the invocation of Collections.emptyList returns a value of type List, which is incompatible with the method processStringList. Thus, in Java SE 7, you must specify the value of the value of the type argument as follows:

processStringList(Collections.<String>emptyList());

This is no longer necessary in Java SE 8. The notion of what is a target type has been expanded to include method arguments, such as the argument to the method processStringList. In this case, processStringList requires an argument of type List

Collections.emptyList() is a generic method similar to the get() method from the question. In Java 7 the print(String string) method is not even applicable to the method invocation thus it doesn't take part in the overload resolution process. Whereas in Java 8 both methods are applicable.

This incompatibility is worth mentioning in the Compatibility Guide for JDK 8.

---

You can check out my answer for a similar question related to overloaded methods resolution Method overload ambiguity with Java 8 ternary conditional and unboxed primitives

According to JLS 15.12.2.5 Choosing the Most Specific Method:

If more than one member method is both accessible and applicable to a method invocation, it is necessary to choose one to provide the descriptor for the run-time method dispatch. The Java programming language uses the rule that the most specific method is chosen.

Then:

One applicable method m1 is more specific than another applicable method m2, for an invocation with argument expressions e1, ..., ek, if any of the following are true:

1. m2 is generic, and m1 is inferred to be more specific than m2 for argument expressions e1, ..., ek by §18.5.4.

2. m2 is not generic, and m1 and m2 are applicable by strict or loose invocation, and where m1 has formal parameter types S1, ..., Sn and m2 has formal parameter types T1, ..., Tn, the type Si is more specific than Ti for argument ei for all i (1 ≤ i ≤ n, n = k).

3. m2 is not generic, and m1 and m2 are applicable by variable arity invocation, and where the first k variable arity parameter types of m1 are S1, ..., Sk and the first k variable arity parameter types of m2 are T1, ..., Tk, the type Si is more specific than Ti for argument ei for all i (1 ≤ i ≤ k). Additionally, if m2 has k+1 parameters, then the k+1'th variable arity parameter type of m1 is a subtype of the k+1'th variable arity parameter type of m2.

The above conditions are the only circumstances under which one method may be more specific than another.

A type S is more specific than a type T for any expression if S <: T (§4.10).

The second of three options matches our case. Since String is a subtype of Object (String <: Object) it is more specific. Thus the method itself is more specific. Following the JLS this method is also strictly more specific and most specific and is chosen by the compiler.

Ical

Thanks for the reference. It's still not clear to me though. Choosing the most specific method is not something that was added in Java 8. Section 15.12.2.5 from Java 7 has similar wording. We could rephrase the question as "Why does Java 7 pick the Object overload? Shouldn't it pick the most specific one?" – Faught

@bayou.io: No, it's not, yours is! As Bogdans comment suggests, this answer overlooks the fact that the important changes are in the type inference algorithm, not in the overloading resolution. – Woodsman

@BogdanCalmac: See bayou.io's answer and my comments on it for an explanation which I think is more correct. – Woodsman

@Woodsman - huh? I think this answer is correct and detailed. of course, any sufficiently complicated answer is indistinguishable from "just because". – Feaster

@bayou.io: Huh? I don't see how I can be any more clear in my critique. Also: This answer refers to the JLS but doesn't really explain much. – Woodsman

@Woodsman you are right, I didn't cover the type inference in much detail although I mentioned that type inference received a significant overhaul in Java 8. I updated my answer with more details providing an example explained in Java Tutorial docs.oracle.com/javase/tutorial/java/generics/… – Ical

@BogdanCalmac please see my updated answer. I covered type inference in more detail. The key point is that the method print(String string) is not even applicable to invocation in Java 7 – Ical

@medvedev1088: The expression new SillyGenericWrapper().get() is (I think) what the JLS call a poly expression, parametrised on the unbound type parameter T. Can you also explain how T is chosen to be String? Or how print(String) is chosen given that T is unbound? I think this secion might explain it together with the section you cite, but I don't really understand how. – Woodsman

@medvedev1088: To show this you must (I think, this is where I don't really understand the JLS): Explain how section number two ("2. m2 is not generic, and m1 and m2 are applicable by strict or loose invocation...") and the last sentence ("A type S is more specific than a type T for any expression if S <: T") applies poly expressions. – Woodsman

@Woodsman You are right this is a poly expression. To answer your question: the compiler first needs to find all applicable methods, both methods are applicable. print(Object) is applicable with inferred type Object, print(String) is applicable with inferred type String. Next step is the compiler finds the most specific method. From JLS this process is independent on invocation context i.e. only method parameters matter. – Ical

@Woodsman thanks for pointing this out. I will try to dig deeper and provide more details in the answer. – Ical

Answer 2

In java7, expressions are interpreted from bottom up (with very few exceptions); the meaning of a sub-expression is kind of "context free". For a method invocation, the types of the arguments are resolved fist; the compiler then uses that information to resolve the meaning of the invocation, for example, to pick a winner among applicable overloaded methods.

In java8, that philosophy does not work anymore, because we expect to use implicit lambda (like x->foo(x)) everywhere; the lambda parameter types are not specified and must be inferred from context. That means, for method invocations, sometimes the method parameter types decide the argument types.

Obviously there's a dilemma if the method is overloaded. Therefore in some cases, it's necessary to resolve method overloading first to pick one winner, before compiling the arguments.

That is a major shift; and some old code like yours will fall victim to incompatibility.

A workaround is to provide a "target typing" to the argument with "casting context"

    print( (Object)new SillyGenericWrapper().get() );

or like @Holger's suggestion, provide type parameter <Object>get() to avoid inference all together.

---

Java method overloading is extremely complicated; the benefit of the complexity is dubious. Remember, overloading is never a necessity - if they are different methods, you can give them different names.

Feaster

I think this answer explains this behaviour the best. I'd like to add a ref to a relevant section in the JLS which discuss and exemplifies how the type inference in different. Quote: "By treating nested generic method invocations as poly expressions, we improve the behavior of inference for nested invocations. For example, the following is illegal in Java SE 7 but legal in Java SE 8: ProcessBuilder b = new ProcessBuilder(Collections.emptyList());" – Woodsman

@Woodsman - thanks. I'm too lazy to go into details (and I don't remember most of it); this whole thing is pretty much magic. – Feaster

The reason I think this explains it the best is that it stresses that the behaviour has more to do with changes in the type inference algorithm than with overloading. – Woodsman

The gist of the details is: Java 7 only did target type inference with an assignment using =, and then only as a kind of "fallback". Also, type inference happened before considering overloads. Java 8 will instead try inference for each possible overload and then pick the most specific. – Heshum

@Heshum - yep. one more detail, java7 #15.12.2.8 also applies to return statements, tho the spec is not very clear on this. (too pedantic; what is wrong with us?:) – Feaster

@Heshum - no, in this case, java8 does overload resolution first, pick one method; then uses that method parameter types for target typing. – Feaster

15.12.2.8 is what I mean by "fallback". It's why e.g. List<Object> x = unmodifiableList(new ArrayList<String>()); fails in Java 7, because the target type is only considered if the inferred type is otherwise unresolved. WRT to "when" exactly Java 8 does inference, I've found it unclear because both sections reference each other. Overload resolution needs inference to determine applicability and inference needs an overload to be chosen for the target type. I'm not sure it's necessarily defined but it would seem inference actually happens during overload resolution now. – Heshum

Answer 3

First of all it has nothing to do with overriding , but it has to deal with overloading.

Jls,. Section 15 provides lot of information on how exactly compiler selects the overloaded method

The most specific method is chosen at compile time; its descriptor determines what method is actually executed at run time.

So when invoking

print(new SillyGenericWrapper().get());

The compiler choose String version over Object because print method that takes String is more specific then the one that takes Object. If there was Integer instead of String then it will get selected.

Moreover if you want to invoke method that takes Object as a parameter then you can assign the return value to the parameter of type object E.g.

public class GenericTypeInference {

    public static void main(String[] args) {
        final SillyGenericWrapper sillyGenericWrapper = new SillyGenericWrapper();
        final Object o = sillyGenericWrapper.get();
        print(o);
        print(sillyGenericWrapper.get());
    }

    private static void print(Object object) {
        System.out.println("Object");
    }

    private static void print(Integer integer) {
        System.out.println("Integer");
    }

    public static class SillyGenericWrapper {
        public <T> T get() {
            return null;
        }
    }
}

It outputs

Object
Integer

The situation starts to become interesting when let say you have 2 valid method definations that are eligible for overloading. E.g.

private static void print(Integer integer) {
    System.out.println("Integer");
}

private static void print(String integer) {
    System.out.println("String");
}

and now if you invoke

print(sillyGenericWrapper.get());

The compiler will have 2 valid method definition to choose from , Hence you will get compilation error because it cannot give preference to one method over the other.

Schoolboy

Of course, you can also just use print(new SillyGenericWrapper().<Object>get()); to invoke print(Object). +1 for pointing to the overload resolution. – Jacy

This answer does not address the question about how the type for get is chosen and why this is different in Java 8. It links to the JLS for Java 7 and uses that as an argument for the behaviour in Java 8, even though the behaviour was different in Java 7. – Woodsman

haven't noticed that I pasted link to Java-7 instead of Java-8 jls. Corrected. Could you please explain what is not clearer – Schoolboy

Since the section in the JLS that you cite is the same for Java 7 and 8 it can clearly not explain why the behaviour is different in the two versions. You draw the wrong conclusions from this section somehow. Otherwise the behaviour would be the one you describe in Java 7 also, but it isn't. – Woodsman

Answer 4

I ran it using Java 1.8.0_40 and got "Object".

If you'll run the following code:

public class GenericTypeInference {

private static final String fmt = "%24s: %s%n";
public static void main(String[] args) {

    print(new SillyGenericWrapper().get());

    Method[] allMethods = SillyGenericWrapper.class.getDeclaredMethods();
    for (Method m : allMethods) {
        System.out.format("%s%n", m.toGenericString());
        System.out.format(fmt, "ReturnType", m.getReturnType());
        System.out.format(fmt, "GenericReturnType", m.getGenericReturnType());   
   }

   private static void print(Object object) {
       System.out.println("Object");
   }

   private static void print(String string) {
       System.out.println("String");
   }

   public static class SillyGenericWrapper {
       public <T> T get() {
           return null;
       }
   }
}

You will see that you get:

Object public T com.xxx.GenericTypeInference$SillyGenericWrapper.get() ReturnType: class java.lang.Object GenericReturnType: T

Which explains why the method overloaded with Object is used and not the String one.

Melidamelilot

Isn't that just shifting the question? The Object override is picked because T is inferred as Object. But now the question is "Why is T inferred as Object?"? – Mudslinging

Did you compile it using Java 8? As that’s what makes the difference. – Jacy

This is not relevant for the question. The erasure of the return type of the method, which is what getReturnType returns, obviously will be Object. – Woodsman