Build go dependencies in separate Docker layer

Question

I'm trying to speed up Docker builds of my Go app. Right now, it's spending maybe 60s just building dependencies (it's a k8s controller, so there are a lot).

One very important constraint: my project depends on private GitHub repos. I do go mod vendor outside docker build, where I have creds for the repos set up.

My Dockerfile right now is roughly:

FROM golang:1.12

WORKDIR /src

COPY . .
RUN go build -mod=vendor
...

Even without having to download the deps, that build takes a while because it rebuilds several hundred packages every docker build.

What I'd like to do is something like:

FROM golang:1.12

WORKDIR /src

# these shouldn't change very often
COPY go.mod go.sum vendor ./
RUN go build -mod=vendor <all dependency packages>

COPY . .
RUN go build -mod=vendor
...

I tried parsing go.mod, but of course that lists modules, not packages. I tried go list but never managed to get a working incantation.

Tammy

Take a look at multistage build from Docker. Another approach is build your dependencies in a different container, publish it , and use it as a base container for your application building container. – Josiejosler

@atayenel I'm using multistage builds, just don't think they're relevant? – Tammy

@atayenel And I'm not sure how base images help. Are you saying I could have a base image that has an old build with perhaps up to date dependencies? Building that base image has the same problem – Tammy

Yes, but you are not updating your deps too much( your words) , you just build it once and next iterations will use it until you update your deps again – Josiejosler

keep an eye open on the process of modularizing K8S. Once that's done, using a local module repository (like Athens or Artifactory) and a central module repository (like GoCenter) wil do magic to your build times. – Counteract

@fons, why not use another builder docker image which is run to process the build. So instead of using docker build, you use docker run to create a build inside docker image. This gives you a chance to save the build cache back to the host and re-use those folders during the next build – Malchus

Answer 1

I've got a nasty hack that seems to work:

FROM golang:1.12

WORKDIR /src

COPY go.mod go.sum ./
COPY vendor/ ./vendor
RUN go build -mod=vendor $(cat deps|grep -v mypackage | grep -v internal)

COPY . .
RUN go build -mod=vendor
...

go list -f '{{join .Deps "\n"}}'  > deps
docker build .

Tammy

May be a bit less of a hack would be to override GOCACHE before building the deps in such a way so that they end up being a part of your Docker build context, then copy them over, and then override this env. var accordingly in the image. Please see this and follow the links. Note that the -x command-line option to the go {build|install} commands makes them chatty about what they are doing (and where they look for files etc). – Thorley

@Thorley that's not a bad idea. I'm running on a Mac though. I assume GOCACHE isn't platform independent? – Tammy

It's dependent in the sense the location of the user's home directory is. But note that the go tool has a built-in default for this variable (whih can be known by running go env or go env GOCACHE), and this default can be overridden by the environment variable with the same name. (It actually works for any variable known by go env). – Thorley

I mean platform-dependent as in "packages compiled in the cache for darwin won't work on Linux". Surely those aren't portable between OSes. – Tammy

I beleive "it's not true either way" ;-) The reason is that the compiled packages are not merely lumped in a common place but are rather separated there by the GOOS and GOARCH under which they were built. It means two things: 1) packages compiled natively on Mac OS won't "spoil" the cache mounted on Linux as the compiler told to build native code there simply won't make use of those cached packages; 2) Packages cross-compiled on MacOS X, can be used in the image. – Thorley

Another problem with this solution is that the go compiler and git are required to be installed in the host OS – Expedient

Answer 2

Docker documentation has a guide specific to Go docker images (Build your Go image).

It works as follows:

# Layer for dependency installation
COPY go.mod go.sum ./
RUN go mod download
# Layer for app build
COPY . .
RUN go build -o main .

Dichasium