Multiple wildcards on a generic methods makes Java compiler (and me!) very confused
Asked Answered
C

3

66

Let's first consider a simple scenario (see complete source on ideone.com):

import java.util.*;

public class TwoListsOfUnknowns {
    static void doNothing(List<?> list1, List<?> list2) { }

    public static void main(String[] args) {
        List<String> list1 = null;
        List<Integer> list2 = null;
        doNothing(list1, list2); // compiles fine!
    }
}

The two wildcards are unrelated, which is why you can call doNothing with a List<String> and a List<Integer>. In other words, the two ? can refer to entirely different types. Hence the following does not compile, which is to be expected (also on ideone.com):

import java.util.*;

public class TwoListsOfUnknowns2 {
    static void doSomethingIllegal(List<?> list1, List<?> list2) {
        list1.addAll(list2); // DOES NOT COMPILE!!!
            // The method addAll(Collection<? extends capture#1-of ?>)
            // in the type List<capture#1-of ?> is not applicable for
            // the arguments (List<capture#2-of ?>)
    }
}

So far so good, but here's where things start to get very confusing (as seen on ideone.com):

import java.util.*;

public class LOLUnknowns1 {
    static void probablyIllegal(List<List<?>> lol, List<?> list) {
        lol.add(list); // this compiles!! how come???
    }
}

The above code compiles for me in Eclipse and on sun-jdk-1.6.0.17 in ideone.com, but should it? Is it not possible that we have a List<List<Integer>> lol and a List<String> list, the analogous two unrelated wildcards situations from TwoListsOfUnknowns?

In fact the following slight modification towards that direction does not compile, which is to be expected (as seen on ideone.com):

import java.util.*;

public class LOLUnknowns2 {
    static void rightfullyIllegal(
            List<List<? extends Number>> lol, List<?> list) {

        lol.add(list); // DOES NOT COMPILE! As expected!!!
            // The method add(List<? extends Number>) in the type
            // List<List<? extends Number>> is not applicable for
            // the arguments (List<capture#1-of ?>)
    }
}

So it looks like the compiler is doing its job, but then we get this (as seen on ideone.com):

import java.util.*;

public class LOLUnknowns3 {
    static void probablyIllegalAgain(
            List<List<? extends Number>> lol, List<? extends Number> list) {

        lol.add(list); // compiles fine!!! how come???
    }
}

Again, we may have e.g. a List<List<Integer>> lol and a List<Float> list, so this shouldn't compile, right?

In fact, let's go back to the simpler LOLUnknowns1 (two unbounded wildcards) and try to see if we can in fact invoke probablyIllegal in any way. Let's try the "easy" case first and choose the same type for the two wildcards (as seen on ideone.com):

import java.util.*;

public class LOLUnknowns1a {
    static void probablyIllegal(List<List<?>> lol, List<?> list) {
        lol.add(list); // this compiles!! how come???
    }

    public static void main(String[] args) {
        List<List<String>> lol = null;
        List<String> list = null;
        probablyIllegal(lol, list); // DOES NOT COMPILE!!
            // The method probablyIllegal(List<List<?>>, List<?>)
            // in the type LOLUnknowns1a is not applicable for the
            // arguments (List<List<String>>, List<String>)
    }
}

This makes no sense! Here we aren't even trying to use two different types, and it doesn't compile! Making it a List<List<Integer>> lol and List<String> list also gives a similar compilation error! In fact, from my experimentation, the only way that the code compiles is if the first argument is an explicit null type (as seen on ideone.com):

import java.util.*;

public class LOLUnknowns1b {
    static void probablyIllegal(List<List<?>> lol, List<?> list) {
        lol.add(list); // this compiles!! how come???
    }

    public static void main(String[] args) {
        List<String> list = null;
        probablyIllegal(null, list); // compiles fine!
            // throws NullPointerException at run-time
    }
}

So the questions are, with regards to LOLUnknowns1, LOLUnknowns1a and LOLUnknowns1b:

  • What types of arguments does probablyIllegal accept?
  • Should lol.add(list); compile at all? Is it typesafe?
  • Is this a compiler bug or am I misunderstanding the capture conversion rules for wildcards?

Appendix A: Double LOL?

In case anyone is curious, this compiles fine (as seen on ideone.com):

import java.util.*;

public class DoubleLOL {
    static void omg2xLOL(List<List<?>> lol1, List<List<?>> lol2) {
        // compiles just fine!!!
        lol1.addAll(lol2);
        lol2.addAll(lol1);
    }
}

Appendix B: Nested wildcards -- what do they really mean???

Further investigation indicates that perhaps multiple wildcards has nothing to do with the problem, but rather a nested wildcard is the source of the confusion.

import java.util.*;

public class IntoTheWild {

    public static void main(String[] args) {
        List<?> list = new ArrayList<String>(); // compiles fine!

        List<List<?>> lol = new ArrayList<List<String>>(); // DOES NOT COMPILE!!!
            // Type mismatch: cannot convert from
            // ArrayList<List<String>> to List<List<?>>
    }
}

So it looks perhaps a List<List<String>> is not a List<List<?>>. In fact, while any List<E> is a List<?>, it doesn't look like any List<List<E>> is a List<List<?>> (as seen on ideone.com):

import java.util.*;

public class IntoTheWild2 {
    static <E> List<?> makeItWild(List<E> list) {
        return list; // compiles fine!
    }
    static <E> List<List<?>> makeItWildLOL(List<List<E>> lol) {
        return lol;  // DOES NOT COMPILE!!!
            // Type mismatch: cannot convert from
            // List<List<E>> to List<List<?>>
    }
}

A new question arises, then: just what is a List<List<?>>?

Centri answered 23/8, 2010 at 10:37 Comment(4)
If anyone wants to dig around the FAQ: angelikalanger.com/GenericsFAQ/JavaGenericsFAQ.html I'm about to start now....Centri
Well, I was exactly on this FAQ. But even with this, I can't figure why this is not a bug.Philina
Related: #1772692Centri
I really don't know if I did a good job, but I've tried to explain why this is like this hereAgeless
C
75

As Appendix B indicates, this has nothing to do with multiple wildcards, but rather, misunderstanding what List<List<?>> really means.

Let's first remind ourselves what it means that Java generics is invariant:

  1. An Integer is a Number
  2. A List<Integer> is NOT a List<Number>
  3. A List<Integer> IS a List<? extends Number>

We now simply apply the same argument to our nested list situation (see appendix for more details):

  1. A List<String> is (captureable by) a List<?>
  2. A List<List<String>> is NOT (captureable by) a List<List<?>>
  3. A List<List<String>> IS (captureable by) a List<? extends List<?>>

With this understanding, all of the snippets in the question can be explained. The confusion arises in (falsely) believing that a type like List<List<?>> can capture types like List<List<String>>, List<List<Integer>>, etc. This is NOT true.

That is, a List<List<?>>:

  • is NOT a list whose elements are lists of some one unknown type.
    • ... that would be a List<? extends List<?>>
  • Instead, it's a list whose elements are lists of ANY type.

Snippets

Here's a snippet to illustrate the above points:

List<List<?>> lolAny = new ArrayList<List<?>>();

lolAny.add(new ArrayList<Integer>());
lolAny.add(new ArrayList<String>());

// lolAny = new ArrayList<List<String>>(); // DOES NOT COMPILE!!

List<? extends List<?>> lolSome;

lolSome = new ArrayList<List<String>>();
lolSome = new ArrayList<List<Integer>>();

More snippets

Here's yet another example with bounded nested wildcard:

List<List<? extends Number>> lolAnyNum = new ArrayList<List<? extends Number>>();
    
lolAnyNum.add(new ArrayList<Integer>());
lolAnyNum.add(new ArrayList<Float>());
// lolAnyNum.add(new ArrayList<String>());     // DOES NOT COMPILE!!

// lolAnyNum = new ArrayList<List<Integer>>(); // DOES NOT COMPILE!!

List<? extends List<? extends Number>> lolSomeNum;
    
lolSomeNum = new ArrayList<List<Integer>>();
lolSomeNum = new ArrayList<List<Float>>();
// lolSomeNum = new ArrayList<List<String>>(); // DOES NOT COMPILE!!

Back to the question

To go back to the snippets in the question, the following behaves as expected (as seen on ideone.com):

public class LOLUnknowns1d {
    static void nowDefinitelyIllegal(List<? extends List<?>> lol, List<?> list) {
        lol.add(list); // DOES NOT COMPILE!!!
            // The method add(capture#1-of ? extends List<?>) in the
            // type List<capture#1-of ? extends List<?>> is not 
            // applicable for the arguments (List<capture#3-of ?>)
    }
    public static void main(String[] args) {
        List<Object> list = null;
        List<List<String>> lolString = null;
        List<List<Integer>> lolInteger = null;

        // these casts are valid
        nowDefinitelyIllegal(lolString, list);
        nowDefinitelyIllegal(lolInteger, list);
    }
}

lol.add(list); is illegal because we may have a List<List<String>> lol and a List<Object> list. In fact, if we comment out the offending statement, the code compiles and that's exactly what we have with the first invocation in main.

All of the probablyIllegal methods in the question, aren't illegal. They are all perfectly legal and typesafe. There is absolutely no bug in the compiler. It is doing exactly what it's supposed to do.


References

Related questions


Appendix: The rules of capture conversion

(This was brought up in the first revision of the answer; it's a worthy supplement to the type invariant argument.)

5.1.10 Capture Conversion

Let G name a generic type declaration with n formal type parameters A1…An with corresponding bounds U1…Un. There exists a capture conversion from G<T1…Tn> to G<S1…Sn>, where, for 1 <= i <= n:

  1. If Ti is a wildcard type argument of the form ? then …
  2. If Ti is a wildcard type argument of the form ? extends Bi, then …
  3. If Ti is a wildcard type argument of the form ? super Bi, then …
  4. Otherwise, Si = Ti.

Capture conversion is not applied recursively.

This section can be confusing, especially with regards to the non-recursive application of the capture conversion (hereby CC), but the key is that not all ? can CC; it depends on where it appears. There is no recursive application in rule 4, but when rules 2 or 3 applies, then the respective Bi may itself be the result of a CC.

Let's work through a few simple examples:

  • List<?> can CC List<String>
    • The ? can CC by rule 1
  • List<? extends Number> can CC List<Integer>
    • The ? can CC by rule 2
    • In applying rule 2, Bi is simply Number
  • List<? extends Number> can NOT CC List<String>
    • The ? can CC by rule 2, but compile time error occurs due to incompatible types

Now let's try some nesting:

  • List<List<?>> can NOT CC List<List<String>>
    • Rule 4 applies, and CC is not recursive, so the ? can NOT CC
  • List<? extends List<?>> can CC List<List<String>>
    • The first ? can CC by rule 2
    • In applying rule 2, Bi is now a List<?>, which can CC List<String>
    • Both ? can CC
  • List<? extends List<? extends Number>> can CC List<List<Integer>>
    • The first ? can CC by rule 2
    • In applying rule 2, Bi is now a List<? extends Number>, which can CC List<Integer>
    • Both ? can CC
  • List<? extends List<? extends Number>> can NOT CC List<List<Integer>>
    • The first ? can CC by rule 2
    • In applying rule 2, Bi is now a List<? extends Number>, which can CC, but gives a compile time error when applied to List<Integer>
    • Both ? can CC

To further illustrate why some ? can CC and others can't, consider the following rule: you can NOT directly instantiate a wildcard type. That is, the following gives a compile time error:

    // WildSnippet1
    new HashMap<?,?>();         // DOES NOT COMPILE!!!
    new HashMap<List<?>, ?>();  // DOES NOT COMPILE!!!
    new HashMap<?, Set<?>>();   // DOES NOT COMPILE!!!

However, the following compiles just fine:

    // WildSnippet2
    new HashMap<List<?>,Set<?>>();            // compiles fine!
    new HashMap<Map<?,?>, Map<?,Map<?,?>>>(); // compiles fine!

The reason WildSnippet2 compiles is because, as explained above, none of the ? can CC. In WildSnippet1, either the K or the V (or both) of the HashMap<K,V> can CC, which makes the direct instantiation through new illegal.

Centri answered 23/8, 2010 at 12:12 Comment(12)
So basically, this isn't a bug but a feature to avoid lost of time during compilation ?Philina
@Colin: Finding the answer to "Why is it designed like this?" would be awesome. I'm looking for that right now.Centri
+1 this explanation confirms my gut feeling, but is more convincing than mine :-) As for the why, I still feel that generic subtyping being invariant is a good enough reason for this.Worthington
I'm lost, if "Capture conversion is not applied recursively" why doesn't this compile : List<? extends List<? extends Number>> lolWhat = new ArrayList<List<String>>(); The second part should be ignored. Did I miss something ?Philina
@Peter: I'm actually not quite sure if my answer is correct, now. I'm getting more and more confused =( ; @Colin: yes, valid point. I just discovered Appendix II within the 5 minutes window of that revision, which is why I didn't have time to investigate. I think my answer is partly wrong. I'm keeping this up to inspire others to investigate.Centri
@Centri I though that "recursive capture conversion" was only applied to MyClass<? extends MySecondClass<?>> but the JLS states that non-wilcarded generics are concerned too. Maybe the recursive part is about something like MyClass<? extends MyClass<?>> then we can talk about recursivity.Philina
@Colin, @Peter: Okay I think I finally got this now. Sorry if I caused confusion with the recursive capture conversion thing (which I'm still not sure what it's all about); I was thoroughly confused myself, but Peter was on the right track with generic type invariance being the issue, and me misunderstanding what List<List<?>> really means.Centri
@Colin: The "no recursive capture" part has nothing to do with this problem. That is another issue (which I also still don't quite understand). This issue, the issues brought up in the question, is quite simple. Java generics is invariant, and nested types can be confusing, but if you put the two together very carefully, everything becomes very clear (at least to me, though I may be missing the point you're trying to bring up; maybe you can elaborate on what you're referring to).Centri
Here is some code : ideone.com/2Yrem . I'm still confused with line 40, why doesn't it compile ? So I wrote the method3 and for line 15 I have a compilation problem. But when I wrote the method4, it worked, how can line 19 work and not 15 ?Philina
@Colin: Nothing special about line 19, lol and list has exactly the same type. A List<List<? extends Number>> and a List<List<? extends Integer>> are two different types. Into the first you can add a List<Float>, into the second you can't. In line 15, you can't do lol = list, but you can lol.addAll(list) (see ideone.com/jOF1v ). Similarly line 40, you can't cast a List<List<? extends Integer>> to a List<List<? extends Number>> since they're two incompatible types.Centri
List<? extends List<? extends Number>> can CC List<List<Integer>>, and the next bullet point is List<? extends List<? extends Number>> can NOT CC List<List<Integer>>. So it can or it can't? The fourth bullet point seems redundant to me.Zerk
'List<? extends List<? extends Number>> can CC List<List<Integer>>' is correct. Second statements must be a typo, something like 'List<? extends List<? extends CharSequence >> can NOT CC List<List<Integer>>' was probably intended.Hypaesthesia
P
2
  • No argument with generics should be accepted. In the case of LOLUnknowns1b the null is accepted as if the first argument was typed as List. For example this does compile :

    List lol = null;
    List<String> list = null;
    probablyIllegal(lol, list);
    
  • IMHO lol.add(list); shouldn't even compile but as lol.add() needs an argument of type List<?> and as list fits in List<?> it works.
    A strange example which make me think of this theory is :

    static void probablyIllegalAgain(List<List<? extends Number>> lol, List<? extends Integer> list) {
        lol.add(list); // compiles fine!!! how come???
    }
    

    lol.add() needs an argument of type List<? extends Number> and list is typed as List<? extends Integer>, it fits in. It won't work if it doesn't match. Same thing for the double LOL, and other nested wildcards, as long as the first capture matches the second one, everything is okay (and souldn't be).

  • Again, I'm not sure but it does really seem like a bug.

  • I'm glad to not be the only one to use lol variables all the time.

Resources :
http://www.angelikalanger.com, a FAQ about generics

EDITs :

  1. Added comment about the Double Lol
  2. And nested wildcards.
Philina answered 23/8, 2010 at 11:29 Comment(0)
F
0

not an expert, but I think I can understand it.

let's change your example to something equivalent, but with more distinguishing types:

static void probablyIllegal(List<Class<?>> x, Class<?> y) {
    x.add(y); // this compiles!! how come???
}

let's change List to [] to be more illuminating:

static void probablyIllegal(Class<?>[] x, Class<?> y) {
    x.add(y); // this compiles!! how come???
}

now, x is not an array of some type of class. it is an array of any type of class. it can contain a Class<String> and a Class<Int>. this cannot be expressed with ordinary type parameter:

static<T> void probablyIllegal(Class<T>[] x  //homogeneous! not the same!

Class<?> is a super type of Class<T> for any T. If we think a type is a set of objects, set Class<?> is the union of all sets of Class<T> for all T. (does it include itselft? I dont know...)

Figge answered 23/8, 2010 at 17:38 Comment(2)
Going from List<T> to T[] is not a valid step, since arrays are covariant and generic is invariant in Java. That is, a String[] is an Object[], but a List<String> is not a List<Object>.Centri
you are right. but I'm just trying to make up some analogies to fool ourselves into accepting it on an intuitive level. it's not like I'm going to study the actually type theory.Figge

© 2022 - 2024 — McMap. All rights reserved.