Working offline with SBT and SNAPSHOT dependencies

Question

I have never been able to work offline with SBT on any of my projects. Now I'm in the middle of a move and my wardrobe-server hosting nexus is offline.

So running say:

sbt "set offline := true" run

I get among other missing deps:

[error] unresolved dependency: se.hedefalk#lift-utils_3.0_2.10;0.2-SNAPSHOT: not found

for the dependency declared as:

"se.hedefalk"       %% ("lift-utils_"+liftEdition) % ("0.2-SNAPSHOT"),

But I do have the dependency in my ivy cache:

$ ll ~/.ivy2/cache/se.hedefalk/lift-utils_3.0_2.10/
total 32
drwxr-xr-x  8 viktor  staff   272B Mar 14 11:36 ./
drwxr-xr-x  5 viktor  staff   170B Mar  8 10:38 ../
drwxr-xr-x  3 viktor  staff   102B Mar 14 11:36 docs/
-rw-r--r--  1 viktor  staff   4.5K Mar  4 15:54 ivy-0.2-SNAPSHOT.xml
-rw-r--r--  1 viktor  staff   4.0K Mar  4 15:54 ivy-0.2-SNAPSHOT.xml.original
-rw-r--r--  1 viktor  staff   1.8K Mar 14 11:36 ivydata-0.2-SNAPSHOT.properties
drwxr-xr-x  3 viktor  staff   102B Mar  8 10:38 jars/
drwxr-xr-x  3 viktor  staff   102B Mar 14 11:36 srcs/

Shouldn't that be enough? That whole set "offline := true" has never ever made any difference for me. The dependency resolution seems to happen before.

I really need to get this working now. I have had problems with this for years, but now I'm in the situation where I simply cannot work at all.

Over here: http://www.scala-sbt.org/0.13.5/docs/Detailed-Topics/Dependency-Management-Flow.html

I read

When offline := true, remote SNAPSHOTs will not be updated by a resolution, even an explicitly requested update. This should effectively support working without a connection to remote repositories. Reproducible examples demonstrating otherwise are appreciated. Obviously, update must have successfully run before going offline.

I probably don't understand this correctly, but is it saying I shouldn't have the problem I have?

I have the same problem using:

> sbt "skip in update := true" run

Using sbt 0.13.5

Edit:

After having my repo online again, resolving all deps and then just setting my wifi to offline I can exit sbt and then run it. However, just running ;reload ;clean ;compile and I'm f***ed again. And If I exit I'm not even able to start sbt at all. Dependency resolution for the project happens before I can for instance try to analyse the dependency graph with dependency-tree or similar. This is highly irritating even if not online if say, trying to understand where a failing transitive dependency comes from.

Physiography

Are you creating a project from scratch offline, or you're trying to run an existing project that has previously been updated? – Natural

Existing project. But I recently started using JDK8. Changed scala version from 2.10.2 to 2.10.4 trying to fix some byte code issues related to that and sbt 0.13.1 to 0.13.5 in this build. Changing back both doesn't help though. – Physiography

Once you do a successful update online, do you still have problem offline? I.e. online ALL snapshots are updated (I think once a day). Offline they are not but I think that is IF it is considered previously successfully resolved. Changing scala, sbt versions perhaps makes sbt reset the "resolved" flag so to speak. – Palembang

I finally started my nexus machine again and I'll try and check just that. I guess as you say that changing scala/sbt versions somwhow reset the resolved state. However, I have new missing deps which is hard to track down. It just feels so broken that I cannot even load an SBT project if not all deps are resolved. Then I can't run dependency-graph to check where they stem from for instance. – Physiography

After having my repo online again, resolving all deps and then just setting my wifi to offline I can at least exit sbt and run it again. However, just running ;reload ;clean ;compile and I'm f***ed again. And If I exit I'm not even able to start sbt at all again. – Physiography

This may be relevant: github.com/sbt/sbt/wiki/… – Jahnke

@Jahnke thanks, yes, very relevant :) Any timeframe on that functionality? – Physiography

+1 for offline mode. I'm utterly amazed that this isn't supported already. So net goes down, depending on any SNAPSHOT and BANG, no dev today for you? – Salver

Answer 1

This seems like a terrible hack, but you can specify your ivy cache as an ivy repository, so that once your dependencies are downloaded, they can be resolved from the cache.

For example, your ~/.sbt/repositories could look like this:

[repositories]
  local
  maven-central
  cache: file://${user.home}/.ivy2/cache, [organisation]/[module]/ivy-[revision].xml, [organisation]/[module]/[type]s/[module]-[revision].[type]

Note: I had to set the ivy and artifact patterns explicitly. Add the local cache below any other repos so they get tried first.

Brittaneybrittani

I do not really understand that !!! isn't it supposed to be the default behavior to resort to the local cache before going on a remote repository? I have the same issue. When i'm offline, i have few dependency that can't get resolved. All thttps://mcmap.net/q/661163/-how-to-set-up-sbt-so-it-grabs-all-necessary-dependencies-when-online-and-let-me-work-offline-then rest of them works fine.However i have those two dependency that can't be resolve if i'm not online. – Estevan

How did you know in which file and what to write in it? – Menstruation

:-\ it almost worked, but finally failed with a couple of hadoop deps. Seems like the cached ivy files are not valid: some "The prefix "e" for element "e:sbtTransformHash" is not bound" errors I think related to that issue – Menstruation

What I had to do is first delete some *.xml.original files in the cache, and then I just removed the <e:sbtTransformHash> tags in the files where they appeared. Then it finally worked for me. Additionally, I wrote the repo config in an "offline-repositories" file and start sbt with "sbt -Dsbt.override.build.repos=true -Dsbt.repository.config=./offline-repositories", this way no bad things happen when I am online. – Piggy

can this be made generic so path resolves for any user? – Urial

Why not using ${user.home} for resolving initial path? – Phillisphilly

Answer 2

Me neither can use it in a one-liner.

But if I start sbt and then in sbt I use set offline := true it's working well.

Litchi

Answer 3

There is a wiki page here: https://github.com/sbt/sbt/wiki/User-Stories:--Offline-mode-and-Dependency-Locking , edited in May, that says that SBT's notion of offline isn't working properly. This may be one reason why your build continues to try to resolve dependencies against remote repositories even when the offline setting is set.

Disobey

Answer 4

Starting from sbt 0.13.7, you can work offline when you use the dependency cache

To make it short:

• add updateOptions := updateOptions.value.withCachedResolution(true) in your project settings (and in all subprojects settings)
• run sbt clean compile once online. It run like before, checking all resolution. But also create and store the resolved dependency graph locally (in ~/.sbt/0.13/dependency).

Then, if you don't change your dependencies, sbt will always use the cached dependency and won't need network (I just tried)

Menstruation