Speeding up SBT-assembly

Question

We have a project where we are building a large number of Scalatra microservices by packaging them using the sbt-assembly plugin, then creating Docker images using the sbt-docker plugin. The process is slow due to the number of micro-services because the assembly plugin has to process all the transitive dependencies for each service. However I note most of the micro-services have very similar dependencies. Is there any way to speed this up?

One limitation is we are doing this on Jenkins (CI) server so we are calling sbt clean at the start so we are not able to take advantage of any caching?

Parishioner

This might help #17191255 Did you try scalatra-sbt's DistPlugin? It creates a directory layout containing the libraries instead of a fat-jar. It may be worth a try if the cause was not the slow dependency resolution process of SBT. – Topnotch

Oh, it seems not to work for 0.13.x... – Topnotch

Answer 1

You could assemble a common dependency jar that has all the dependencies all your microservices need, which you would only need to build once. Then you could assemble jars for each of the microservices without any of the dependencies.

See "Splitting your project and deps JARs".

Mikkanen