Java interoperability woes with Scala generics and boxing
Asked Answered
D

1

8

Suppose I've got this Scala trait:

trait UnitThingy {
  def x(): Unit
}

Providing a Java implementation is easy enough:

import scala.runtime.BoxedUnit;

public class JUnitThingy implements UnitThingy {
  public void x() {
    return;
  }
}

Now let's start with a generic trait:

trait Foo[A] {
  def x(): A
}

trait Bar extends Foo[Unit]

The approach above won't work, since the unit x returns is now boxed, but the workaround is easy enough:

import scala.runtime.BoxedUnit;

public class JBar implements Bar {
  public BoxedUnit x() {
    return BoxedUnit.UNIT;
  }
}

Now suppose I've got an implementation with x defined on the Scala side:

trait Baz extends Foo[Unit] {
  def x(): Unit = ()
}

I know I can't see this x from Java, so I define my own:

import scala.runtime.BoxedUnit;

public class JBaz implements Baz {
  public BoxedUnit x() {
    return BoxedUnit.UNIT;
  }
}

But that blows up:

[error] .../JBaz.java:3: error: JBaz is not abstract and does not override abstract method x() in Baz
[error] public class JBaz implements Baz {
[error]        ^
[error] /home/travis/tmp/so/js/newsutff/JBaz.java:4: error: x() in JBaz cannot implement x() in Baz
[error]   public BoxedUnit x() {
[error]                    ^
[error]   return type BoxedUnit is not compatible with void

And if I try the abstract-class-that-delegates-to-super-trait trick:

abstract class Qux extends Baz {
  override def x() = super.x()
}

And then:

public class JQux extends Qux {}

It's even worse:

[error] /home/travis/tmp/so/js/newsutff/JQux.java:1: error: JQux is not abstract and does not override abstract method x() in Foo
[error] public class JQux extends Qux {}
[error]        ^

(Note that this definition of JQux would work just fine if Baz didn't extend Foo[Unit].)

If you look at what javap says about Qux, it's just weird:

public abstract class Qux implements Baz {
  public void x();
  public java.lang.Object x();
  public Qux();
}

I think the problems here with both Baz and Qux have to be scalac bugs, but is there a workaround? I don't really care about the Baz part, but is there any way I can inherit from Qux in Java?

Dermatome answered 30/10, 2014 at 0:28 Comment(0)
L
8

They aren't scalac bugs; it's that the Scala compiler is working hard on your behalf to paper over the difference between procedures and methods, and the Java compiler is not.

For efficiency and Java compatibility, methods that return Unit non-generically are actually implemented as procedures (i.e. return type is void). Then the generic implementation is implemented by calling the void version and returning BoxedUnit.

public abstract class Qux implements Baz {
  public void x();
    Code:
       0: aload_0       
       1: invokestatic  #17            // Method Baz$class.x:(LBaz;)V
       4: return        

  public java.lang.Object x();
    Code:
       0: aload_0       
       1: invokevirtual #22            // Method x:()V
       4: getstatic     #28            // Field scala/runtime/BoxedUnit.UNIT:Lscala/runtime/BoxedUnit;
       7: areturn

The problem is that while javac will do the same thing for you with specific vs. generic Object-derived return types, it doesn't understand the Object-void crossover.

That's an explanation. There is a workaround, though it complicates the Scala hierarchy:

trait Bazz[U <: Unit] extends Bar[Unit] {
  def x() = ().asInstanceOf[U]    // Must go here, not in Baz!
}
trait Baz extends Bazz[Unit] {}

Now you have forced Scala to consider the possibility of some not-exactly-Unit return type, so it keeps BoxedUnit for the return; and Baz throws away that possibility, but it doesn't generate a new void x() to confuse Java.

This is fragile, to say the least. Fixing it may be a job for both the Java and Scala teams, though: Java is not happy as long as the BoxedUnit version is there; it gets actively annoyed by the void version. (You can generate an abstract class with both by inheriting from Foo twice; since it doesn't work the details are unimportant.) Scala might be able to do it alone by emitting altered bytecode that has an extra BoxedUnit method everywhere Java expects it...not sure.

Laxative answered 30/10, 2014 at 1:58 Comment(25)
Maybe "bug" is the wrong word—I mean something more like "betrayal of a pretty reasonable expectation". The BoxedUnit.UNIT trick lets me implement a generic Unit method, so why does adding one level of inheritance break that? It's not like it makes the method less generic in any real way.Dermatome
It's not that there's one level of inheritance, it's that there's a new public void x() method, and javac is not happy about it. It thinks that that is the one that is implementing the trait, even though there is a BoxedUnit version around. It's looking for a signature of BoxedUnit x(), which is what it would generate itself.Laxative
(There is a BoxedUnit version around, but it is typed as Object, I mean.)Laxative
+1, that's really clever, and something I'd never, ever actually use.Dermatome
It is a bug. Groovy doesn't have these problems and yet can do functional programming and handle actors and everything with GPars and is 100% interoperable with Java.Pydna
@Pydna - But Groovy is incredibly slow, weakly typed, and how is this relevant to the question? A design decision that differs from Groovy is not a "bug", it's just a different decision.Laxative
Thats pure FUD. Groovy can run as fast as Java and can be static typed static typed or dynamic typed (try reading up on your languages). The 'design decision' (as you call it) in Scala makes it's an interoperability issue. As an example, I used another non-Java Java language to show proper Java interoperability; Groovy has 100% Java interoperability and functional programming and doesn't have issues like this. Bad Java interoperability in a Java language is seen by most in the Java community as a 'bug'.Pydna
@Pydna - Can you link to something like the Computer Languages Benchmark Game set of tests for Groovy vs. Java to back up your claim? (Yes, Groovy can be statically typed, but only if you add boilerplate; called Groovy code might be dynamic and give you all sorts of stupid errors. And still it is slow from the tests I've seen--very often 2x slower.) Also, this is a question about whether it is a Scala bug. Making every drawback when viewed from Java be a bug in your language is a design decision.Laxative
Its a known by most of the community. It's why Google has it as part of their SDK. Also you don't need boiler plate to static type... you can do it by adding a bean with one line of code to configure. And no it's not slow... only slow if you are writing VERY VERY bad code (which is just about every test I have ever seen as well). I can make Scala run extremely slow too if I am not as versed in it and try to write tests to prove a point. It seems you already have a preconceived notion and I don't really care to convert you to Groovy. In fact, people like you should stay on Scala.Pydna
And if that STILL doesn't answer your question and you want to troll further... "Using @CompileStatic, you will ensure that your classes are statically compiled, meaning that all method calls are resolved at compile time, leading to dramatically improved performance, very close, if not equal, to that of Java (depending on how you write your code, as usual, because idiomatic Groovy might not always be the fastest implementation)." ( melix.github.io/blog/2014/06/grooid2.html )Pydna
@Pydna - Yes, that's very nice rhetoric. Can you please post benchmarks?Laxative
@Pydna - Just in case you think I'm being pedantic just to annoy: Clojure can make similar claims to Groovy, you know. But look at the benchmarks: benchmarksgame.alioth.debian.org/u64q/… It's almost always slower than Scala (by 50% or more) and has more code than Scala, which reflects that while you might be able to be fast-ish in Clojure, it's painful. I'd like better evidence than "a blog said so" that the situation is meaningfully different for Groovy. All benchmarks are flawed, but often less than opinion is.Laxative
You have to understand Groovy though. When you do @CompileStatic, you are LITERALLY writing Java; Groovy is just a layer on top of Java with convenience methods. Clojure and Scala are NOT. Groovy allows Java INLINE because underneath, IT IS JAVA!!! Clojure and Scala are NOT!Pydna
@Pydna - No, you're not literally writing Java. See for instance groovy.329449.n5.nabble.com/… where an innocent "mistake" results in a 20x performance penalty.Laxative
No not true. I can still use Groovy convenience methods. But to the original point, I have full Java INTEROPERABILTY and Java speed... unlike Scala. Thank you for bringing me full circle. :)Pydna
@Pydna - I am glad you're happy with Groovy, and you are fortunate to not need to actually care whether Groovy is really as fast as Java, so you can rely upon warm fuzzy feelings instead of benchmarks.Laxative
@Rex - I'm glad you can ignore facts and rely on sarcasm instead of interoperability and speed.Pydna
@Rex- Sarcasm is the last resort of those, who when faced with logic, cling to their emotional choices. Take some time, re-evaluate and see if I'm right. Hey I'f I'm so wrong, why are so many companies using Groovy over Scala? Why is Groovy part of the Android toolkit instead of Scala? There are logical answers... you just don't like them and want to be snarky. I would prefer we address this as engineers but it's your choice. As I already said, I would prefer people like you NOT use Groovy. You are a detriment to the community and have shown you don't understand it or it's benefits.Pydna
@Pydna - You already ignored the division problem. I could bring up others, but why, if you're just going to ignore it and reiterate your first point? And you continue to make claims without supporting evidence. "Groovy is as fast as Java." Still no benchmarks. "People use Groovy over Scala." Some do, sure--different people prefer different tradeoffs. But did you check the number of questions about Scala and Groovy on SO, or ranking on TIOBE, or anything? You cannot have sound argumentation with flawed premises.Laxative
@Rex You NEVER pointed to a division problem in Groovy (check the posts) hence I did not address one as (a) one doesnt exists and (b) you didn't present an argument for one in groovy. So (c) logically, what am I ignoring? Again, if you want a benchmark, go make one. Scala has already stated the are not 100% interoperable with Java. Their website states this, the fact they AREN'T interoperable proves this. You seem to want to argue something that Scala acknowledges... and you do it badly. Because you cannot present logical arguments without emotion.Pydna
@Pydna - On my post dated "Jun 5 at 19:35", I link to a severe Groovy performance problem that turns out to have been caused by innocently assuming that "/ 2" means the same thing that it does in Java. Also, my point all along has been that Scala isn't promising to be fully interoperable; you were the one that flat-out stated that some case of non-interoperability was a bug (rather than a consequence of design choices).Laxative
@Rex read the post better; an answer is provided that aligns with what I repeat to you over and over. Like I keep saying, Groovy is just a layer on top of Java offering functional programming and convenience methods. You can default to Java at any time. Scala lacks interoperability because it literally is trying to create a new language. You obviously have a bias in the fact that you want to argue only points that you wish to see and don't ask questions. If you really want to know if it has better interoperability over Scala, try it for 2 weeks.Pydna
@Rex Well if you are saying that you are making the same point that Scala's lack of interoperability is due to the fact that it isn't meant to be 100% interoperable with Java and if you want Java interoperability and functional programming to switch to Groovy, then yes. We do agree and have made the same point I guess.Pydna
@Pydna - You make no distinction between what I agree with and what I don't agree with in what you say (you've done it again), which makes it pointless to "discuss" further.Laxative
Let us continue this discussion in chat.Laxative

© 2022 - 2024 — McMap. All rights reserved.