Moving a package-private class—should I consider that binary incompatible?

Question

Because of an issue with package name aux under Windows, I am moving a helper class within the package hierarchy of my library from

de.sciss.scalainterpreter.aux

to

de.sciss.scalainterpreter

The class is private to the library, i.e. private[scalainterpreter] object Helper.

Now using the Typesafe Migration-Manager, obviously it reports that the change is not compatible:

Found 2 binary incompatibiities
===============================
 * class de.sciss.scalainterpreter.aux.Helper does not have a correspondent
   in new version
 * object de.sciss.scalainterpreter.aux.Helper does not have a correspondent
   in new version

But I suspect that if client code does not call into either object, the interfaces are still compatible, and thus I can use a minor version increase to indicate the change, and allow those two versions to be used interchangeably.

Correct?

Watt

Answer 1

It's easy to see how inlining can break client code, since the inlined code essentially bleeds into the client interface. This example really asks for a linkage error; we can experiment and do things like javap | grep Helper, but at some level you have to let scalac do its job.

package lib {

  object Lib {
    //import util.Helper
    @inline def result = Helper.help
  }

  //package util {

  private [lib] object Helper {
    @inline def help = "Does this help?"
  }
//}
}

Sample innocently bystanding client:

package client

object Test {
  import lib.Lib
  def main(args: Array[String]) {
    println(Lib.result)
  }
}

Changing package of package-private class:

$ scala -cp "classes;target" client.Test
Does this help?

apm@halyard ~/tmp/taking-it-private
$ vi lib.scala

apm@halyard ~/tmp/taking-it-private
$ rm -rf classes/*

apm@halyard ~/tmp/taking-it-private
$ smalac -d classes -optimise lib.scala 

apm@halyard ~/tmp/taking-it-private
$ smala -cp "classes;target" client.Test
java.lang.ClassNotFoundException: lib.util.Helper$

Javap shows why. [Namely, the call is inlined but it still wants to init the module.]

I haven't followed the discussions, but for example there are links at: https://github.com/scala/scala/pull/1133 and other discussions on the ML about what expectations about binary compatibility are valid. https://groups.google.com/forum/?fromgroups=#!topic/scala-internals/sJ-xnWL_8PE

Tournai

Good point, I hadn't thought about inlining; seems intricate enough. Luckily, in my case, there aren't inline annotations and I didn't compile with -optimize. – Watt

This answer is a response to "I suspect that if client code does not call into either object,...", namely, you can't reason from source compat to binary compat. This is one example where the machinery is visible, and one example suffices. You could find others, with your other question about final val consts or implicits. Otherwise, your question is tautological: either yes of course you're fine, or of course not. – Tournai

Answer 2

You are not specifying if Helper was already package private before the move. So I'll treat both cases:

• If it was already package private:

  I suspect that the migration manager reports an incompatibility only because it must stay conservative: packages are open in scala (like in java), which means that client code might very well define a class package scalainterpreter. So by moving Helper, you would indeed break that class.

  However let's be pragmatic: de.sciss.scalainterpreter.aux is your package (and so should be their sub-packages), and nobody should define their own classes there. With this additional prerequiste, moving Helper is indeed a binary compatible change toward client scala code.

  As for client java code, it's a bit different because even if Helper is package private, its visibility is still public as far as the JVM is concerned, and thus the java compiler will happily let client code access Helper (thus client java code might very well already access Helper, despite it being declared package private).

• If it was not package private before the move:

  Well, tough luck. Client code could very well already access Helper, and the move will certainly break that. As a side note, you can employ a little trick to make the change source-compatible, but alas not binary-compatible. Just add the following file:

  package de.sciss
  
  package object scalainterpreter {
    object aux {
      val Helper = _root_.de.sciss.scalainterpreter.Helper
    }
  }
  

With the above, you can still access Helper as de.sciss.scalainterpreter.aux.Helper, and it still compiles under windows (unlike defining a package aux, which does not compile because of the reserved meaning as a file name). But again, this is not binary compatible, only source compatible.

Farthingale

Thanks. Sorry if it was unclear—yes, Helper has been package private before, nothing changed regarding its visibility. – Watt

Then you're on the safe side, provided you had no inline annotation, as mentioned by user1296806. The inline annotation is generally unwarranted anyway, as the jitter can most of the time inline your hot spots (especially on a server VM). – Militarist

@user1296806 Considering the sheer size of the library, a few hundred occurence of the inline annotations (I found less than 100 in the library itself) is very low. I would have expected more. It's not like inlining is never useful. – Militarist

Answer 3

Simply put, no reason why it wouldn't be. Linkage happens around signatures; since the object in question is scoped to the compilation unit, clients cannot (or rather, should not) be using it, and binary compatibility is therefore not an issue.

Lucila