sbt assembly: deduplicate module-info.class

Question

I get the following error when assembling my uber jar:

java.lang.RuntimeException: deduplicate: different file contents found in the following: [error] /Users/jake.stone/.ivy2/cache/org.bouncycastle/bcprov-jdk15on/jars/bcprov-jdk15on-1.61.jar:module-info.class [error] /Users/jake.stone/.ivy2/cache/javax.xml.bind/jaxb-api/jars/jaxb-api-2.3.1.jar:module-info.class

I am not up to date with java technology, but assume I cannot simply discard one of these classes.

Can someone tell me what mergeStrategy I can use to safely compile the uber jar?

Cornwell

Answer 1

The answer depends on your environment and what you want to achieve.

JDK 8

I had the same problem with a project using JDK 8. JDK 8 does not use the file module-info.class so it is safe to discard the file.

Add the following to your build.sbt:

assemblyMergeStrategy in assembly := {
  case x if x.endsWith("module-info.class") => MergeStrategy.discard
  case x =>
    val oldStrategy = (assemblyMergeStrategy in assembly).value
    oldStrategy(x)
}

This simply discards the files.

JDK 11

If you use JDK 11 with an end user project (not a library) then it should also be safe as if you create the uber-jar all classes are included and no external dependencies are needed. Just tested it with a short test (not thoroughly enough to say it is always safe).

If you use JDK 11 and create a library then it is better not to create an uber-jar. In this case a dropping of the module-info.class will most likely create a jar that will not work. In this case simply depend on the libraries.

Disenthrone

As the files have moved to META-INF/versions/9, I had to change the discard case to case x if x.endsWith("/module-info.class"). – Narcoma

Answer 2

The module-info.class file has been moved in many libraries. Here is the updated solution

assembly / assemblyMergeStrategy := {
    case PathList("module-info.class") => MergeStrategy.last
    case path if path.endsWith("/module-info.class") => MergeStrategy.last
    case x =>
      val oldStrategy = (assembly / assemblyMergeStrategy).value
      oldStrategy(x)
  }

Skin

would not be better to concat them? – Forborne

Answer 3

Each jar file containing this file has information about it's own module and dependencies, you can find more info here

I haven't tried this but I think the proper strategy is to concat the content of each jar file so all modules will be included on this file (instead of getting just the last one as some of the previous answers)

assemblyMergeStrategy in assembly := {
  case x if x.endsWith("module-info.class") => MergeStrategy.concat
  case x =>
    val oldStrategy = (assemblyMergeStrategy in assembly).value
    oldStrategy(x)
}

The main difference with previous answer is in the merge strategy:

MergeStrategy.concat

Forborne