Simple use of Scala collections from Java not compiling with 2.11

Question

So I've got this super exciting Java class:

import scala.collection.immutable.Stream;

public class EmptyStreamFactory {
  public static Stream<String> createEmptyStringStream() {
    return Stream.<String>empty();
  }
}

Compiles just fine with the 2.10.4 scala-library.jar on the classpath (or 2.9.2, for what that's worth). Now I try it with 2.11:

EmptyStreamFactory.java:5: error: incompatible types
    return Stream.<String>empty();
                               ^
  required: Stream<String>
  found:    GenTraversable
1 error

How does this make any sense at all? At a glance the only difference that could be remotely relevant is the fact that Stream.Empty no longer extends Serializable in 2.11, but I don't see how that could cause this problem. The same thing happens with List, etc.

There's an easy workaround—you can just cast to the appropriate type—but I'd like to understand what's going on here.

(I'm on Oracle's JDK, version 1.7.0_67.)

Erkan

Stream.Empty still extends Serializable, because it extends Stream. The removed Serializable on Empty was superfluous, so this can't be the source of your issue. – Trying

Answer 1

The static forwarder for the bridge method isn't marked as a bridge method itself, and java for whatever reason prefers the one returning GenTraversable as it has two to choose from.

classOf[scala.collection.immutable.Stream[_]].getMethods filterNot 
  (_.isBridge) filter (_.getName == "empty") foreach println
public static scala.collection.immutable.Stream scala.collection.immutable.Stream.empty()
public static scala.collection.GenTraversable scala.collection.immutable.Stream.empty()

You can't overload on return type in java the language so it's anyone's guess what the compiler will do when it encounters it. I don't know if it's even specified, although it may be.

In general you can't call collections methods from java, and this has been judged wontfix.

Edit: re "still don't understand what changed in 2.11 to make this happen", here is an initial batch of candidates:

% git log --no-merges --oneline --grep=forwarder v2.10.4..v2.11.2
532ef331eb (pull/3868/head) Restore reporter forwarders in CompilationUnit
b724201383 Rip out reporting indirection from CompilationUnit
98216be3f3 Move reporting logic into Reporting trait
653c404d7f (pull/3493/head) SI-3452 GenBCode version of the static-forwarder signature fix
640e279500 SI-3452 A better fix for static forwarder generic sigs
f8d80ea26a SI-3452 Correct Java generic signatures for mixins, static forwarders
51ec62a8c3 (pull/3480/head) SI-6948 Make the Abstract* classes public.
731ed385de SI-8134 SI-5954 Fix companions in package object under separate comp.
3cc99d7b4a (pull/3103/head) Collections library tidying and deprecation.  Separate parts are listed below.
5d29697365 Flesh out the Delambdafy phase.
6e2cadb8bd (pull/2951/head) SI-7847 Static forwarders for case apply/unapply
9733f56c87 (pull/1173/head) Fixes SI-4996.

You're not going to find it by looking at the library code, that much is certain. It's a change in the compiler.

Exocrine

This isn't you talking about Scala? from the link: I played with it until it got too tedious. – Polydactyl

Ha ha, looks like martin and I agree on something as long as "it" is a type parameter. – Exocrine

Thanks. Still don't understand what changed in 2.11 to make this happen, but oh well. Would you suggest a specific workaround? – Erkan

Sorry, my only suggestions would be in the vein of "don't expect this to work." – Exocrine

Actually, IIRC from last night, the last thing it will try with two abstract members w/same erasure is to take the most specific return type. OK, that's the end of 15.12.2.5. – Polydactyl

Answer 2

What I learned today is that Java is happy to ignore an extraneous type arg (JLS 15.12.2.1, fine print at the end of the section).

This rule stems from issues of compatibility and principles of substitutability.

Apparently, some rules stem from principled reasoning, others from practical issues, and the occasional rule has dual parentage.

apm@mara:~/tmp$ javap -classpath ~/scala-2.11.2/lib/scala-library.jar scala.collection.immutable.Stream | grep empty
  public static <A extends java/lang/Object> scala.collection.immutable.Stream<A> empty();
  public static scala.collection.GenTraversable empty();
apm@mara:~/tmp$ javap -classpath ~/scala-2.10.4/lib/scala-library.jar scala.collection.immutable.Stream | grep empty
  public static <A extends java/lang/Object> scala.collection.immutable.Stream<A> empty();
  public static <A extends java/lang/Object> scala.collection.immutable.Stream<A> empty();

It looks like the forwarder to the bridge method was fixed.

This compiles:

import scala.collection.immutable.Stream;
import scala.collection.immutable.Stream$;

public class EmptyStreamFactory {
    public static Stream<String> createEmptyStringStream() {
        return Stream$.MODULE$.<String>empty();
    }
}

I would need a three-day weekend to re-read the spec on overloading in Java.

Maybe because Stream is abstract, the rule for most-specific return type rule kicks in.

Polydactyl

Thanks for the extra clue, but I still don't understand what's going on. – Erkan

@TravisBrown The second signature comes from scala-lang.org/api/2.10.4/… but wasn't generated to produce GenTraversable, the upper bound. Now it is, and Java overloading selects it. For some reason, it selects it on Stream but not on Stream$. – Polydactyl

On Stream$, the GenTraversable returning method is a bridge method. On Stream, it isn't. – Exocrine

@extempore Thanks, I noticed it last night but it didn't register. It's like the brain has to operate exitingErasure(thunk) so to speak in a different mode. – Polydactyl