Why sbt runs dependency resolution every time after clean?

Question

SBT runs dependency resolution every time after clean even if project dependency management configuration hasn't changed. This is time consuming when running on CI server.

But documentation says:

1. Normally, if no dependency management configuration has changed since the last successful resolution and the retrieved files are still present, sbt does not ask Ivy to perform resolution.

How can I stop sbt from doing dependency resolution every time I build project with sbt clean publish-local ?

Update

I've discovered that sbt also runs resolution when I enter in interactive mode with sbt.

Update2

As @Ezhik pointed if I can preserve target/resolution-cache then sbt will not resolve dependencies after clean. So I tried to move resolution-cache out from target dir:

ivyConfiguration <<= (externalResolvers, ivyPaths, offline, checksums, appConfiguration, target, streams) map { (rs, paths, off, check, app, t, s) =>
        val resCacheDir = t / ".." / "resolution-cache"
        new InlineIvyConfiguration(paths, rs, Nil, Nil, off, Option(lock(app)), check, Some(resCacheDir), s.log)
      }

Now with this code in Build.scala resolution cache is placed in project root and is therefore preserved after clean, but resolution is being done anyway. So I assume this approach is wrong or insufficient.

Barham

Answer 1

Because of directory target/resolution-cache that contains Ivy reports. It is obviously that you remove all target content while clean operation.

IMHO You must point it in your project to somewhere out from target if you want to preserve resolution state.

Updated.

vs SBT.0.12.4.RC1

1. Find where resolution-cache is used - in IvyConfiguration

2. Inspect where IvyConfiguration located - in project scope

   > inspect ivy-configuration
   [info] Task: sbt.IvyConfiguration
   [info] Description:
   [info]  General dependency management (Ivy) settings, such as the resolvers and paths to use.
   [info] Provided by:
   [info]  {file:/home/ezh/projects/sbt/}xsbt/*:ivy-configuration
   [info] Dependencies:
   [info]  xsbt/*:offline
   

3. Fix it in build.sbt.

   ivyConfiguration <<= (ivyConfiguration, baseDirectory) map {
     case (c: InlineIvyConfiguration, b) => import c._
       new InlineIvyConfiguration(paths, resolvers, otherResolvers, moduleConfigurations,
        localOnly, lock, checksums, resolutionCacheDir.map(_ => b / "123"), log)
     case (other, _) => other // something unknown
   }
   

4 Test... Ups... resolution is still active... Investigate. ->

target/scala-2.10/cache/default-920e5d/global/update/output cache contain pointers to resolution-cache :)

1. Fix it.

   cacheDirectory <<= baseDirectory / "234"
   

Test. Got it. Resolution is skipped.

Summary changes for required configuration:

ivyConfiguration <<= (ivyConfiguration, baseDirectory) map {
  case (c: InlineIvyConfiguration, b) => import c._
    new InlineIvyConfiguration(paths, resolvers, otherResolvers, moduleConfigurations,
     localOnly, lock, checksums, resolutionCacheDir.map(_ => b / "123"), log)
  case (other, _) => other // something unknown
}
cacheDirectory <<= baseDirectory / "234"

---

vs SBT.0.13.x

@deprecated("Use the cacheDirectory provided by streams.", "0.13.0")

https://github.com/sbt/sbt/issues/1208

Ambrosio

With your own hands? SBT is very expressive. OK special for you :))) And for me as a funny case. – Ambrosio

+1, great answer @Ezhik. Sadly this solution will not work with sbt 0.13, unfortunately. cacheDirectory has been deprecated (and as far as the dependency resolution workaround is concerned, cacheDirectory has been effectively removed). I have yet to find a workable solution on 0.13, frustrating having sbt check for updates on every single clean/compile. – Topology

@virtualeyes: any luck coming up with a working solution for 0.13.x? – Vasyuta

Answer 2

Might be that you have SNAPSHOT dependencies. They are subject to change anytime so must be resolved on every run. You can suppress this with

    offline := true

Implausibility

I'm aware of snapshots. It's not the case. Sbt runs resolution for all dependencies available. – Barham

Sorry where should I apply this line of code build.sbt? Could you please provide some context... – Cheese

@Cheese run set offline := true in your sbt command line (REPL) OR run sbt with sbt "set offline := true" run. You could in theory also add this to your build.sbt, but then you would always have to edit your build file to toggle offline mode. – Implausibility

Answer 3

This works for me on 0.13.1.

cleanKeepFiles ++= Seq("resolution-cache", "streams").map(target.value / _)

Friesen

works for me in 0.13.2 , you need to add that line to all subprojects individually – Addieaddiego

Answer 4

You can prevent clean from deleting certain files with this setting:

// path should be adapted for newer sbt versions
cleanKeepFiles <+= cacheDirectory / "update"

Hoon