What's the magic behind ScalaFX to make OpenJDK 9+ actually work?

Question

Environment:

• OpenJDK 64-Bit Server VM Zulu12.2+3-CA (build 12.0.1+12, mixed mode, sharing)
• Scala 2.12.7
• Windows 10 Professional, X86_64
• IntelliJ IDEA 2019.1.3 (Ultimate Edition)

I checked out the scalafx-hello-world from GitHub, built and ran it in IntelliJ and it worked all fine. Here quickly the significant application implementation:

package hello

import scalafx.application.JFXApp
import scalafx.application.JFXApp.PrimaryStage
import scalafx.geometry.Insets
import scalafx.scene.Scene
import scalafx.scene.effect.DropShadow
import scalafx.scene.layout.HBox
import scalafx.scene.paint.Color._
import scalafx.scene.paint._
import scalafx.scene.text.Text

object ScalaFXHelloWorld extends JFXApp {

  stage = new PrimaryStage {
    //    initStyle(StageStyle.Unified)
    title = "ScalaFX Hello World"
    scene = new Scene {
      fill = Color.rgb(38, 38, 38)
      content = new HBox {
        padding = Insets(50, 80, 50, 80)
        children = Seq(
          new Text {
            text = "Scala"
            style = "-fx-font: normal bold 100pt sans-serif"
            fill = new LinearGradient(
              endX = 0,
              stops = Stops(Red, DarkRed))
          },
          new Text {
            text = "FX"
            style = "-fx-font: italic bold 100pt sans-serif"
            fill = new LinearGradient(
              endX = 0,
              stops = Stops(White, DarkGray)
            )
            effect = new DropShadow {
              color = DarkGray
              radius = 15
              spread = 0.25
            }
          }
        )
      }
    }

  }
}

EDIT: My build.sbt:

// Name of the project
name := "ScalaFX Hello World"

// Project version
version := "11-R16"

// Version of Scala used by the project
scalaVersion := "2.12.7"

// Add dependency on ScalaFX library
libraryDependencies += "org.scalafx" %% "scalafx" % "11-R16"
resolvers += Resolver.sonatypeRepo("snapshots")

scalacOptions ++= Seq("-unchecked", "-deprecation", "-Xcheckinit", "-encoding", "utf8", "-feature")

// Fork a new JVM for 'run' and 'test:run', to avoid JavaFX double initialization problems
fork := true

// Determine OS version of JavaFX binaries
lazy val osName = System.getProperty("os.name") match {
  case n if n.startsWith("Linux") => "linux"
  case n if n.startsWith("Mac") => "mac"
  case n if n.startsWith("Windows") => "win"
  case _ => throw new Exception("Unknown platform!")
}

// Add JavaFX dependencies
lazy val javaFXModules = Seq("base", "controls", "fxml", "graphics", "media", "swing", "web")
libraryDependencies ++= javaFXModules.map( m=>
  "org.openjfx" % s"javafx-$m" % "11" classifier osName
)

After that, I changed the implementation to:

package hello

import javafx.application.Application
import javafx.scene.Scene
import javafx.scene.control.Label
import javafx.stage.Stage

class ScalaFXHelloWorld extends Application {
  override def start(stage: Stage): Unit = {
    stage.setTitle("Does it work?")
    stage.setScene(new Scene(
      new Label("It works!")
    ))
    stage.show()
  }
}

object ScalaFXHelloWorld {
  def main(args: Array[String]): Unit = {
    Application.launch(classOf[ScalaFXHelloWorld], args: _*)
  }
}

Here I get the following error:

Exception in Application start method
java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplicationWithArgs(LauncherImpl.java:464)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication(LauncherImpl.java:363)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
    at java.base/sun.launcher.LauncherHelper$FXHelper.main(LauncherHelper.java:1051)
Caused by: java.lang.RuntimeException: Exception in Application start method
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication1(LauncherImpl.java:900)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication$2(LauncherImpl.java:195)
    at java.base/java.lang.Thread.run(Thread.java:835)
Caused by: java.lang.IllegalAccessError: superclass access check failed: class com.sun.javafx.scene.control.ControlHelper (in unnamed module @0x40ac0fa0) cannot access class com.sun.javafx.scene.layout.RegionHelper (in module javafx.graphics) because module javafx.graphics does not export com.sun.javafx.scene.layout to unnamed module @0x40ac0fa0
    at java.base/java.lang.ClassLoader.defineClass1(Native Method)
    at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1016)
    at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:151)
    at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:802)
    at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:700)
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:623)
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
    at javafx.scene.control.Control.<clinit>(Control.java:86)
    at hello.ScalaFXHelloWorld.start(ScalaFXHelloWorld.scala:39)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication1$9(LauncherImpl.java:846)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runAndWait$12(PlatformImpl.java:455)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$10(PlatformImpl.java:428)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:389)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$11(PlatformImpl.java:427)
    at javafx.graphics/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:96)
    at javafx.graphics/com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
    at javafx.graphics/com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(WinApplication.java:174)
    ... 1 more
Exception running application hello.ScalaFXHelloWorld

Now my question is: What does ScalaFX that the module problem does not occur?

Lurleen

So far as I can tell, your code looks fine. Can you post the contents of your SBT build file (or equivalent)? – Yount

@MikeAllen I just added the build.sbt - but I did not change the one of ScalaFX example. – Lurleen

I need to give that a test. Just a couple of observations in the meantime: (1) try setting scalaVersion to "2.12.8" or "2.13.0", just in case, since compatibility with Java 9+ is still evolving and a more recent Scala release might be better; (2) I notice you're using ZuluFX, which includes OpenJFX 12, but you're also downloading OpenJFX 11 modules as dependent libraries - try commenting out the libraryDependencies statement in your build.sbt. – Yount

Actually, I can't find ZuluFX 12 on Azul's web-site. They only seem to support ZuluFX (OpenJDK & OpenJFX) 8 & 11. Can you check the version number? – Yount

@MikeAllen There I put the wrong information: it is actually Azul's Zulu 12 (I edited my environment in the original post.) – Lurleen

@MikeAllen I could provide you with a full IDEA project for both versions if you like and tell me how and where. What I did was: I cloned the ScalaFX example, tried it (without any modification) and it worked. Then I only modified the single file I posted here and did not modify anything else. That's why I do not understand what is going on. – Lurleen

The IDEA project isn't too important. It ought to be possible to run the project directly from SBT, by issuing the command sbt run from the command line in the project's root directory. However, I get the error: Error: JavaFX runtime components are missing, and are required to run this application with your build file, JDK, OS & sources. (I'm using SBT 1.2.8.) I've tried different OpenJFX releases too (including 11.0.2 and 12.0.1, instead of 11). – Yount

Let me look into this some more... – Yount

Let us continue this discussion in chat. – Yount

Answer 1

I've not been able to exactly reproduce your problem, but I have been able to get a project that uses JavaFX-only (that is, it doesn't make use of ScalaFX) to build and run.

Here's what I'm using (everything else is specified in the build file):

• Zulu OpenJDK 11
• SBT 1.2.8

(I did try using Zulu OpenJDK 12 to build and run the project, and that worked too. However, it's probably best if you use the version of OpenJFX that matches the JDK.)

When I tried your original sources and build.sbt, I encountered the following error when executing an sbt run command from the command line:

D:\src\javafx11>sbt run
[info] Loading global plugins from {my home directory}\.sbt\1.0\plugins
[info] Loading project definition from D:\src\javafx11\project
[info] Loading settings for project javafx11 from build.sbt ...
[info] Set current project to JavaFX 11 Hello World (in build file:/D:/src/javafx11/)
[info] Running (fork) hello.ScalaFXHelloWorld
[error] Error: JavaFX runtime components are missing, and are required to run this application
[error] Nonzero exit code returned from runner: 1
[error] (Compile / run) Nonzero exit code returned from runner: 1
[error] Total time: 1 s, completed Aug 11, 2019, 3:17:07 PM

as I mentioned in my original comments to your question.

I thought that was odd because the code compiled, which meant that the compiler was able to find the JavaFX runtime just fine.

I then tried running the program without forking, by commenting out the fork := true in the build file. Guess what? The program ran without error!

I may be missing something, regarding using SBT with JDK versions 9+, but this indicated that SBT was somehow not running the forked process correctly. I could force the forked process to run correctly by adding the following to the end of the build file:

val fs = File.separator
val fxRoot = s"${sys.props("user.home")}${fs}.ivy2${fs}cache${fs}org.openjfx${fs}javafx-"
val fxPaths = javaFXModules.map {m =>
  s"$fxRoot$m${fs}jars${fs}javafx-$m-11-$osName.jar"
}
javaOptions ++= Seq(
  "--module-path", fxPaths.mkString(";"),
  "--add-modules", "ALL-MODULE-PATH"
)

This works by adding the downloaded ivy-managed JavaFX jar files to Java's module path. However, this is not a good solution for running standalone applications. It may be possible for the sbt-native-packager to provide the necessary environment for the completed application to run, but I haven't tried that.

I've posted the complete solution on GitHub

Let me know whether this helps. In the meantime, I'll look into SBT's support for JDK 9+ modules to see whether there is a simpler solution...

UPDATE:

I have raised an issue (#4941) with the SBT team to look into this in more detail.

UPDATE 2

I patched an issue that stopped the solution from working on Linux. Perform a git pull to update the sources.

UPDATE 3

I should also mention that it's best to have IntelliJ run the application using SBT, which keeps things simple and ensures that the application's environment is properly configured.

To do this, got to the IntelliJ Run menu, and select the Edit Configurations... option. Click the + button at the top left corner of the dialog, select sbt Task" from the list under **Add New Configuration, then configure as follows:

This will compile and build the application first, if required.

Note: The _VM parameters are for running SBT, and do not relate to how SBT runs your forked application.

(You can also add SBT run configurations to test your code, as well.)

Yount

Thanks a lot for your effort! There is only one question left: Is there a way to debug the application within IntelliJ? The debug mode seems not to use sbt and therefore cannot launch the application ... – Lurleen

@Lurleen Yes, you can debug a run configuration. In the IntelliJ Run menu, the top two items are to run & debug, respectively, the last run configuration that you executed. You can pick a specific run configuration to debug via the Run menu's Debug... option, which allows you to pick the run configuration you want to debug. Let me know if you have any further questions... – Yount

I see that the application is launched when I click on the debug icon but it does not stop at breakpoints for any reason. Have you tried this? – Lurleen

@Lurleen Yes. Are the breakpoints in test code, by any chance? If so, you'll have to debug a run configuration that runs your tests. – Yount

No, they are - for example - on line 20 of HelloWorld.scala. – Lurleen

@Lurleen Hmmm... You're right. It appears you can only debug SBT tasks if fork := true isn't defined (or if fork := false is specified). That shouldn't be a problem provided you don't use the SBT shell, such that each run task is performed in a separate SBT_/_JVM process. Of course, you might also get a different run trajectory by not forking the run process. I'll keep an eye on this... – Yount

I just filed a blunt support request with JetBrains asking them why I cannot run and debug my application with the standard means of IntelliJ. Maybe they come up with a solution. If SBT build and run my application and IntelliJ needs some hacks to debug it (with or without SBT) I am fine for now. I really hope all this was way easier and programmers could concentrate on software development :-) – Lurleen

@Lurleen Agreed. Thanks for reporting that! – Yount

The only answer I got is I have to use --add-exports and --add-modules. Neither do I how nor why: Sacla has no concept for those modules, does it? – Lurleen

Answer 2

Adding to Jonathan Crosmer's answer:

The reason that naming the class and the object differently works is because the Java launcher actually has special behaviour in place if the main class extends javafx.application.Application. If you have the Java sources available, the relevant code can be found in JAVA_HOME/lib/src.zip/java.base/sun/launcher/LauncherHelper.java. In particular there are two methods which are of interest:

public static Class<?> checkAndLoadMain(boolean, int ,String)

//In nested class FXHelper
private static void setFXLaunchParameters(String, int)

The first methods has a check in place that looks if the main class extends javafx.application.Application. If it does, this method replaces the main class with the nested class FXHelper, which has its own public static void main(String[] args).

The second method, which is directly called by the first method, attempts to load the JavaFX runtime. However, the way it does this is by first loading the module javafx.graphics via java.lang.ModuleLayer.boot().findModule(JAVAFX_GRAPHICS_MODULE_NAME). If this call fails, Java will complain about not having found the JavaFX runtime and then immediatly exit via System.exit(1).

Going back to SBT and Scala, some other details are in play. First, if both the main object and the class extending javafx.application.Application have the same name, the Scala compiler will generate a class file which both extends Application and has a public static void main(...). That means that the special behaviour described above will be triggered and the Java launcher will try to load the JavaFX runtime as a module. Since SBT currently has no notion about modules, the JavaFX runtime will not be on the module path and the call to findModule(...) will fail.

On the other hand, if the main object has a different name from the main class, the Scala compiler will place public static void main(...) in a class which does not extend Application, which in turn means that the main() method will execute normally.

Before we go on, we should note that while SBT did not put the JavaFX runtime on the module path, it DID in fact put it on the classpath. That means that the JavaFX classes are visible to JVM, they just can not be loaded as a module. After all

A modular JAR file is like an ordinary JAR file in all possible ways, except that it also includes a module-info.class file in its root directory.

(from The State of the Module System)

However, if a method happens to call, let's say Application.launch(...), Java will happily load javafx.application.Application from the classpath. Application.launch(...) will similarly have access to the rest of JavaFX and everything works out.

That is also the reason why running a JavaFX app without forking works. In that case SBT will always invoke public static void main(...) directly, which means that no special behaviours from the java launcher are triggered and the JavaFX runtime will be found on the classpath.

---

Here is a snippet to see the above behaviour in action:

Main.scala:

object Main {
  def main(args: Array[String]): Unit = {
    /*
    Try to load the JavaFX runtime as a module. This is what happens if the main class extends
    javafx.application.Application.
     */
    val foundModule = ModuleLayer.boot().findModule("javafx.graphics").isPresent
    println("ModuleLayer.boot().findModule(\"javafx.graphics\").isPresent = " + foundModule) // false

    /*
    Try to load javafx.application.Application directly, bypassing the module system. This is what happens if you
    call Application.launch(...)
     */
    var foundClass = false
    try{
      Class.forName("javafx.application.Application")
      foundClass = true
    }catch {
      case e: ClassNotFoundException => foundClass = false
    }
    println("Class.forName(\"javafx.application.Application\") = " + foundClass) //true
  }
}

build.sbt:

name := "JavaFXLoadTest"

version := "0.1"

scalaVersion := "2.13.2"

libraryDependencies += "org.openjfx" % "javafx-controls" % "14"

fork := true

Parsec

There's some interesting detective work here, for sure, but let's be clear about one thing: SBT doesn't generate anything from the sources—it's just a build system. It's the Scala compiler that generates code. I'll take a look into this further... – Yount

Bravo for figuring this out! It's ironic that the culprit is named FXHelper :) – Dryfoos

I hope that one day Java and Scala get their act together enough to where building and deploying a JavaFX app is not such a pain! Those are all great technologies once you get them working, but there are definitely some confusing issues to overcome to get off the ground. – Dryfoos

Note also this discussussion of this in pure Java context: mail.openjdk.java.net/pipermail/openjfx-dev/2018-June/… – Diatonic

Answer 3

I ran across this same exact problem and found a disturbingly odd and easy solution. tldr; make the main class have a different name from the JavaFX Application class. First an example:

import javafx.application.Application
import javafx.event.ActionEvent
import javafx.event.EventHandler
import javafx.scene.Scene
import javafx.scene.control.Button
import javafx.scene.layout.StackPane
import javafx.stage.Stage

object HelloWorld {
  def main(args: Array[String]): Unit = {
    Application.launch(classOf[HelloWorld], args: _*)
  }
}

// Note: Application class name must be different than main class name to avoid JavaFX path initialization problems!  Try renaming HelloWorld -> HelloWorld2
class HelloWorld extends Application {
  override def start(primaryStage: Stage): Unit = {
    primaryStage.setTitle("Hello World!")
    val btn = new Button
    btn.setText("Say 'Hello World'")
    btn.setOnAction(new EventHandler[ActionEvent]() {
      override def handle(event: ActionEvent): Unit = {
        System.out.println("Hello World!")
      }
    })
    val root = new StackPane
    root.getChildren.add(btn)
    primaryStage.setScene(new Scene(root, 300, 250))
    primaryStage.show()
  }
}

The code as written above throws the exception from the original question. If I rename the class HelloWorld to HelloWorld2 (keeping object HelloWorld, and changing the launch call to classOf[HelloWorld2]), it runs fine. I suspect this is the "magic" that makes ScalaFX work as well, because it is wrapping the JavaFX Application in its own JFXApp type, creating a hidden Application class.

Why does it work? I'm not completely sure, but when running each piece of code in IntelliJ using a standard Run config (right-click HelloWorld and "run HelloWorld.main()") , then in the output clicking "/home/jonathan/.jdks/openjdk-14.0.1/bin/java ..." to expand it shows a command that includes "--add-modules javafx.base,javafx.graphics ", among other things. In the second version, with the renamed HelloWorld2 app, the command does not include this. I can't figure out how IntelliJ has decided to make the command different, but I can only speculate it has something to do with inferring it is a JavaFX app and trying to be helpful by automatically adding "--add-modules "...? In any case, the modules list doesn't include all the modules needed, so for example creating a button requires "javafx.controls", and you get the error. But when the main class doesn't match the Application name, whatever magic inference it does gets turned off, and the standard classpath from the build.sbt just works.

Fun follow up: if I run the application from the sbt shell using sbt run, then the pattern is the same (HelloWorld fails, but renaming the application class fixes it), but the error message is the more-straightforward-but-still-unhelpful "Error: JavaFX runtime components are missing, and are required to run this application". So maybe not entirely an IntelliJ problem, but something to do with JavaFX and Jigsaw? Anyway it's a mystery, but at least we have an easy solution.

Dryfoos

I might have found why this works. Please check out my answer – Parsec

This is odd, because companion objects always have different names to their associated classes. The Scala compiler decorates the name of the object, HelloWorld, to be type HelloWorld$ (that is, it appends a $), but the class remains HelloWorld, so they should appear different to Java all the time. I'll look into this some more... – Yount

The Scala compiler will always place a static version of object methods in the HelloWorld class, which just forwards the call to HelloWorld$. This enables them to be used from Java code without having to write identifiers with $. The Scala tour calls this static forwarding. That means that if you have a companion class to your main object, it will have a public static void main(...), even though the actual content of the method will be in the class file with $. – Parsec