How to link classes from JDK into scaladoc-generated doc?

Question

I'm trying to link classes from the JDK into the scaladoc-generated doc. I've used the -doc-external-doc option of scaladoc 2.10.1 but without success.

I'm using -doc-external-doc:/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/rt.jar#http://docs.oracle.com/javase/7/docs/api/, but I get links such as index.html#java.io.File instead of index.html?java/io/File.html. Seems like this option only works for scaladoc-generated doc.

Did I miss an option in scaladoc or should I fill a feature request?

I've configured sbt as follows:

 scalacOptions in (Compile,doc) += "-doc-external-doc:/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/rt.jar#http://docs.oracle.com/javase/7/docs/api"

Note: I've seen the Opts.doc.externalAPI util in the upcoming sbt 0.13. I think a nice addition (not sure if it's possible) would be to pass a ModuleID instead of a File. The util would figure out which file corresponds to the ModuleID.

Willner

You should report to scaladoc. – Tetrapod

Is there a command line for this outside sbt? Following @jsuereth's answer, sbt just passes the options to scaladoc and if it doesn't support the feature, sbt doesn't either. – Sorilda

Answer 1

I use sbt 0.13.5.

There's no out-of-the-box way to have the feature of having Javadoc links inside scaladoc. And as my understanding goes, it's not sbt's fault, but the way scaladoc works. As Josh pointed out in his comment You should report to scaladoc.

There's however a workaround I came up with - postprocess the doc-generated scaladoc so the Java URLs get replaced to form proper Javadoc links.

The file scaladoc.sbt should be placed inside a sbt project and whenever doc task gets executed, the postprocessing via fixJavaLinksTask task kicks in.

NOTE There are lots of hardcoded paths so use it with caution (aka do the polishing however you see fit).

import scala.util.matching.Regex.Match

autoAPIMappings := true

// builds -doc-external-doc
apiMappings += (
    file("/Library/Java/JavaVirtualMachines/jdk1.8.0_11.jdk/Contents/Home/jre/lib/rt.jar") -> 
    url("http://docs.oracle.com/javase/8/docs/api")
)

lazy val fixJavaLinksTask = taskKey[Unit](
    "Fix Java links - replace #java.io.File with ?java/io/File.html"
)

fixJavaLinksTask := {
  println("Fixing Java links")
  val t = (target in (Compile, doc)).value
  (t ** "*.html").get.filter(hasJavadocApiLink).foreach { f => 
    println("fixing " + f)
    val newContent = javadocApiLink.replaceAllIn(IO.read(f), fixJavaLinks)
    IO.write(f, newContent)
  }
}

val fixJavaLinks: Match => String = m =>
    m.group(1) + "?" + m.group(2).replace(".", "/") + ".html"

val javadocApiLink = """\"(http://docs\.oracle\.com/javase/8/docs/api/index\.html)#([^"]*)\"""".r

def hasJavadocApiLink(f: File): Boolean = (javadocApiLink findFirstIn IO.read(f)).nonEmpty

fixJavaLinksTask <<= fixJavaLinksTask triggeredBy (doc in Compile)

Sorilda

Answer 2

I took the answer by @jacek-laskowski and modified it so that it avoid hard-coded strings and could be used for any number of Java libraries, not just the standard one.

Edit: the location of rt.jar is now determined from the runtime using sun.boot.class.path and does not have to be hard coded.

The only thing you need to modify is the map, which I have called externalJavadocMap in the following:

import scala.util.matching.Regex
import scala.util.matching.Regex.Match

val externalJavadocMap = Map(
  "owlapi" -> "http://owlcs.github.io/owlapi/apidocs_4_0_2/index.html"
)

/*
 * The rt.jar file is located in the path stored in the sun.boot.class.path system property.
 * See the Oracle documentation at http://docs.oracle.com/javase/6/docs/technotes/tools/findingclasses.html.
 */
val rtJar: String = System.getProperty("sun.boot.class.path").split(java.io.File.pathSeparator).collectFirst {
  case str: String if str.endsWith(java.io.File.separator + "rt.jar") => str
}.get // fail hard if not found

val javaApiUrl: String = "http://docs.oracle.com/javase/8/docs/api/index.html"

val allExternalJavadocLinks: Seq[String] = javaApiUrl +: externalJavadocMap.values.toSeq

def javadocLinkRegex(javadocURL: String): Regex = ("""\"(\Q""" + javadocURL + """\E)#([^"]*)\"""").r

def hasJavadocLink(f: File): Boolean = allExternalJavadocLinks exists {
  javadocURL: String => 
    (javadocLinkRegex(javadocURL) findFirstIn IO.read(f)).nonEmpty
}

val fixJavaLinks: Match => String = m =>
  m.group(1) + "?" + m.group(2).replace(".", "/") + ".html"

/* You can print the classpath with `show compile:fullClasspath` in the SBT REPL.
 * From that list you can find the name of the jar for the managed dependency.
 */
lazy val documentationSettings = Seq(
  apiMappings ++= {
    // Lookup the path to jar from the classpath
    val classpath = (fullClasspath in Compile).value
    def findJar(nameBeginsWith: String): File = {
      classpath.find { attributed: Attributed[File] => (attributed.data ** s"$nameBeginsWith*.jar").get.nonEmpty }.get.data // fail hard if not found
    }
    // Define external documentation paths
    (externalJavadocMap map {
      case (name, javadocURL) => findJar(name) -> url(javadocURL)
    }) + (file(rtJar) -> url(javaApiUrl))
  },
  // Override the task to fix the links to JavaDoc
  doc in Compile <<= (doc in Compile) map {
    target: File =>
      (target ** "*.html").get.filter(hasJavadocLink).foreach { f => 
        //println(s"Fixing $f.")
        val newContent: String = allExternalJavadocLinks.foldLeft(IO.read(f)) {
          case (oldContent: String, javadocURL: String) =>
            javadocLinkRegex(javadocURL).replaceAllIn(oldContent, fixJavaLinks)
        }
        IO.write(f, newContent)
      }
      target
  }
)

I am using SBT 0.13.8.

Pentecostal

I tried adding your code to my SBT build file, but ScalaDoc still cannot seem to find JDK classes. I probably did something stupid – this should be added to build.sbt, right? – Travelled

Yes, it should be added to build.sbt. What is the value of System.getProperty("sun.boot.class.path")? You can check by typing console from the SBT REPL to drop into the Scala REPL. Are running on Windows or Linux or Mac? Do you get any warnings when running the doc task? – Pentecostal

@Hawk Perhaps most importantly, did you add documentationSettings to your project's settings? For example, something like lazy val myProject = (project in file("myProject")).settings(documentationSettings: _*) in build.sbt. – Pentecostal

@AndrewL Adding documentationSettings to the project settings fixes it; as I had expected, I was doing something tremendously wrong. Thanks! – Travelled

Someone should pluginize this. <hint> <hint> – Moorwort

re "someone should pluginize this", maybe it should be part of github.com/ThoughtWorksInc/sbt-api-mappings — someone might open an issue there – Millisecond

Alas, as of JDK 9+, System.getProperty("sun.boot.class.path") returns null, due to Java's modularization, so this approach will not work unless you're using JDK 8. – Suppurative