How to make a SBT task depend on a module defined in the same SBT project?

Question

I have module A and module B in a multi-module SBT project. I want to write a resource generator task for module B that invokes code from module A. One way to do this is to pull all the code from module A under project/ but that is unfeasible as module A is massive and I would like to keep it where it is (see https://mcmap.net/q/1509487/-sbt-how-to-package-an-instance-of-a-class-as-a-jar). How do I do this in SBT?

Secondly, is it possible to get rid of module B altogether i.e. I want the resource generator task for module A actually invoke code from module A but module A is a root module and does not live in SBT's project?

Note: This question is not a duplicate of this: Defining sbt task that invokes method from project code? since that one resolves to moving the code into SBT's project which is specifically something I am seeking to avoid here.

Heliacal

Answer 1

Based on @0__ answer above, here is my simplified version:

  /**
    * Util to run the main of a sub-module from within SBT
    *
    * @param cmd The cmd to run with the main class with args (if any)
    * @param module The sub-module
    */
  def runModuleMain(cmd: String, module: Reference) = Def.task {
    val log = streams.value.log
    log.info(s"Running $cmd ...")
    val classPath = (fullClasspath in Runtime in module).value.files
    val opt = ForkOptions(bootJars = classPath, outputStrategy = Some(LoggedOutput(log)))
    val res = Fork.scala(config = opt, arguments = cmd.split(' '))
    require(res == 0, s"$cmd exited with code $res")
  }

You can then invoke it as:

resourceGenerators in Compile += Def.taskDyn {
  val dest = (resourceManaged in Compile).value
  IO.createDirectory(dest)
  runModuleMain(
    cmd = "com.mycompany.foo.ResourceGen $dest $arg2 $arg3 ...",
    module = $referenceToModule // submodule containing com.mycompany.foo.ResourceGen
  ).taskValue
  dest.listFiles()
}

There is one more way to do this:

resourceGenerators in Compile += Def.taskDyn {
  val dest = (resourceManaged in Compile).value
  IO.createDirectory(dest)      
  Def.task {
    val cmd = "com.mycompany.foo.ResourceGen $dest $arg2 $arg3 ..."        
    (runMain in Compile).toTask(" " + $cmd).value 
    dest.listFiles()
  }
}.taskValue

Heliacal

Answer 2

I think I'm doing something like your first part in the following project: My module gen is the equivalent of your module A, and my module core is the equivalent of your module B. Without testing, the structure is roughly as follows:

// taking inspiration from
// http://stackoverflow.com/questions/11509843/
lazy val ugenGenerator = TaskKey[Seq[File]]("ugen-generate", "Generate UGen class files")

lazy val gen = Project(id = "gen", base = file("gen")) ...

lazy val core = Project(id = "core", base = file("core"))
  .settings(
    sourceGenerators in Compile <+= ugenGenerator in Compile,
    ugenGenerator in Compile := {
      val src   = (sourceManaged       in Compile        ).value
      val cp    = (dependencyClasspath in Runtime in gen ).value
      val st    = streams.value
      runUGenGenerator(description.value, outputDir = src, 
        cp = cp.files, log = st.log)
    }
  )

def runUGenGenerator(name: String, outputDir: File, cp: Seq[File],
                    log: Logger): Seq[File] = {
  val mainClass   = "my.class.from.Gen"
  val tmp         = java.io.File.createTempFile("sources", ".txt")
  val os          = new java.io.FileOutputStream(tmp)

  log.info(s"Generating UGen source code in $outputDir for $name")

  try {
    val outs  = CustomOutput(os)
    val fOpt  = ForkOptions(javaHome = None, outputStrategy = Some(outs), bootJars = cp,
        workingDirectory = None, connectInput = false)
    val res: Int = Fork.scala(config = fOpt,
      arguments = mainClass :: "-d" :: outputDir.getAbsolutePath :: Nil)

    if (res != 0) {
      sys.error(s"UGen class file generator failed with exit code $res")
    }
  } finally {
    os.close()
  }
  val sources = scala.io.Source.fromFile(tmp).getLines().map(file).toList
  tmp.delete()
  sources
}

This works in sbt 0.13, but I haven't had time to figure out why it doesn't work in 1.x.

By the way, how do I write sourceGenerators in Compile <+= ugenGenerator in Compile without deprecated syntax?

Aleshia

I think you can get rid of lhs <+= foo with lhs += foo.value Anyway I am getting this when I try this: Error: Could not find or load main class scala.tools.nsc.MainGenericRunner java.lang.IllegalArgumentException: requirement failed: com.github.pathikrit.ResourceGenerator – Heliacal

Does this mean I have to add the Scala compiler itself as a dependency somewhere??? – Heliacal

Not sure, but my 'gen' module indeed adds the compiler: "org.scala-lang" % "scala-compiler" % scalaVersion.value (don't remember why) – Aleshia

Hmm, one more thing - module A itself loads some other stuff from its own resources. Invoking module A using Fork.scala seems to throw a NPE when I do getClass.getResourceAsStream(foo) from within module A. How do I get around this? Do I need to pass in the resourced to ForkOption somehow?? – Heliacal

Not sure, but the docs suggest that you may need fullClasspath instead. – Aleshia

Yup that works! One more question - I ran into one another way while searching for an answer: reformatcode.com/code/scala/… . It uses runMain.toTask(s" $cmd") - I could not make it work at all - but I wonder what are your thoughts on that vs. this approach? – Heliacal