Slow gradle build in Docker. Caching gradle build

Question

I am doing university project where we need to run multiple Spring Boot applications at once.

I had already configured multi-stage build with gradle docker image and then run app in openjdk:jre image.

Here is my Dockerfile:

FROM gradle:5.3.0-jdk11-slim as builder
USER root
WORKDIR /usr/src/java-code
COPY . /usr/src/java-code/

RUN gradle bootJar

FROM openjdk:11-jre-slim
EXPOSE 8080
WORKDIR /usr/src/java-app
COPY --from=builder /usr/src/java-code/build/libs/*.jar ./app.jar
ENTRYPOINT ["java", "-jar", "app.jar"]

I am building and running everything with docker-compose. Part of docker-compose:

 website_server:
    build: website-server
    image: website-server:latest
    container_name: "website-server"
    ports:
      - "81:8080"

Of course first build take ages. Docker is pulling all it's dependencies. And I am okay with that.

Everything is working ok for now but every little change in code causes around 1 min build time for one app.

Part of build log: docker-compose up --build

Step 1/10 : FROM gradle:5.3.0-jdk11-slim as builder
 ---> 668e92a5b906
Step 2/10 : USER root
 ---> Using cache
 ---> dac9a962d8b6
Step 3/10 : WORKDIR /usr/src/java-code
 ---> Using cache
 ---> e3f4528347f1
Step 4/10 : COPY . /usr/src/java-code/
 ---> Using cache
 ---> 52b136a280a2
Step 5/10 : RUN gradle bootJar
 ---> Running in 88a5ac812ac8

Welcome to Gradle 5.3!

Here are the highlights of this release:
 - Feature variants AKA "optional dependencies"
 - Type-safe accessors in Kotlin precompiled script plugins
 - Gradle Module Metadata 1.0

For more details see https://docs.gradle.org/5.3/release-notes.html

Starting a Gradle Daemon (subsequent builds will be faster)
> Task :compileJava
> Task :processResources
> Task :classes
> Task :bootJar

BUILD SUCCESSFUL in 48s
3 actionable tasks: 3 executed
Removing intermediate container 88a5ac812ac8
 ---> 4f9beba838ed
Step 6/10 : FROM openjdk:11-jre-slim
 ---> 0e452dba629c
Step 7/10 : EXPOSE 8080
 ---> Using cache
 ---> d5519e55d690
Step 8/10 : WORKDIR /usr/src/java-app
 ---> Using cache
 ---> 196f1321db2c
Step 9/10 : COPY --from=builder /usr/src/java-code/build/libs/*.jar ./app.jar
 ---> d101eefa2487
Step 10/10 : ENTRYPOINT ["java", "-jar", "app.jar"]
 ---> Running in ad02f0497c8f
Removing intermediate container ad02f0497c8f
 ---> 0c63eeef8c8e
Successfully built 0c63eeef8c8e
Successfully tagged website-server:latest

Every time it freezes after Starting a Gradle Daemon (subsequent builds will be faster)

I was thinking about adding volume with cached gradle dependencies but I don't know if that is core of the problem. Also i could't find good examples for that.

Is there any way to speed up the build?

Mcgaw

I'm not really familiar with Java and Gradle, but isn't it the same behavior as in the local development? I mean if you made some changes to your code you need to recompile the project as to apply the changes also to the runtime. Maybe what you meant is that the Gradle recompiles all of the project instead of only changed parts? – Firer

Posted Dockerfile works ok but the problem is speed. Building locally it takes up ~8 seconds and in Docker ~1 to 1,5 minutes. I was wondering if there is a way to speed up docker build. – Mcgaw

Answer 1

Build takes a lot of time because Gradle every time the Docker image is built downloads all the plugins and dependencies.

There is no way to mount a volume at the image build time. But it is possible to introduce new stage that will download all dependencies and will be cached as Docker image layer.

FROM gradle:5.6.4-jdk11 as cache
RUN mkdir -p /home/gradle/cache_home
ENV GRADLE_USER_HOME /home/gradle/cache_home
COPY build.gradle /home/gradle/java-code/
WORKDIR /home/gradle/java-code
RUN gradle clean build -i --stacktrace

FROM gradle:5.6.4-jdk11 as builder
COPY --from=cache /home/gradle/cache_home /home/gradle/.gradle
COPY . /usr/src/java-code/
WORKDIR /usr/src/java-code
RUN gradle bootJar -i --stacktrace

FROM openjdk:11-jre-slim
EXPOSE 8080
USER root
WORKDIR /usr/src/java-app
COPY --from=builder /usr/src/java-code/build/libs/*.jar ./app.jar
ENTRYPOINT ["java", "-jar", "app.jar"]

Gradle plugin and dependency cache is located in $GRADLE_USER_HOME/caches. GRADLE_USER_HOME must be set to something different than /home/gradle/.gradle. /home/gradle/.gradle in parent Gradle Docker image is defined as volume and is erased after each image layer.

In the sample code GRADLE_USER_HOME is set to /home/gradle/cache_home.

In the builder stage Gradle cache is copied to avoid downloading the dependencies again: COPY --from=cache /home/gradle/cache_home /home/gradle/.gradle.

The stage cache will be rebuilt only when build.gradle is changed. When Java classes are changes, cached image layer with all dependencies is reused.

This modifications can reduce the build time but more clean way of building Docker images with Java applications is Jib by Google. There is a Jib Gradle plugin that allows to build container images for Java applications without manually creating Dockerfile. Building image with application and running the container is similar to:

gradle clean build jib
docker-compose up

Leyden

Multi-stage build with a stage including only build.gradle from context is definitely the way to go. By copying only build.gradle in cache you ensure dependencies will only be downloaded once if the Gradle build file does not change (Docker will re-use the cache) – Huynh

@Evgeniy Khyst how does it suppose to work when cache stage is using build task that for spring boot is invoking also bootJar and you won't have sources with main class which will yield org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':bootJar' - Main class name has not been configured and it could not be resolved after downloading dependencies. Thanks – Ninnetta

Ok, actually I made the mistake here. If you set the mainClassName in booJar task you are good to go. When there are sources task is resolving the main class name from them if no property is set explicitly. Additionally if using kotlin even if main class is com.xyz.Main.kt file I needed to use com.xyz.MainKt name. – Ninnetta

@Saris I had the same issue, and was able to exclude the bootJar in the cache phase with RUN gradle clean build -i --stacktrace -x bootJar without having to worry about specifying the mainClassName. – Cyanamide

@Evgeniy Khyst excellent solution, thank you! – Leeland

Answer 2

Docker caches its images in "layers." Each command that you run is a layer. Each change that is detected in a given layer invalidates the layers that come after it. If the cache is invalidated, then the invalidated layers must be built from scratch, including dependencies.

I would suggest splitting your build steps. Have a previous layer which only copies the dependency specification into the image, then runs a command which will result in Gradle downloading the dependencies. After that's complete, copy your source into the same location where you just did that, and run the real build.

This way, the previous layers will be invalidated only when the gradle files change.

I haven't done this with Java/Gradle, but I have followed the same pattern with a Rust project, guided by this blog post.

Anabal

Answer 3

You can try and use BuildKit (now activated by default in the latest docker-compose 1.25)

See "Speed up your java application Docker images build with BuildKit!" from Aboullaite Med.

(This was for maven, but the same idea applies to gradle)

let's consider the following Dockerfile:

FROM maven:3.6.1-jdk-11-slim AS build  
USER MYUSER  
RUN mvn clean package  

Modifying the second line always invalidate maven cache due to false dependency, which exposes inefficient caching issue.

BuildKit solves this limitation by introducing the concurrent build graph solver, which can run build steps in parallel and optimize out commands that don’t have an impact on the final result.

Additionally, Buildkit tracks only the updates made to the files between repeated build invocations that optimize the access to the local source files. Thus, there is no need to wait for local files to be read or uploaded before the work can begin.

Toreutic

The problem doesn't have to do with building Docker images, but running commands in the Dockerfile. I think it's the caching issue. I've tried caching but it still downloads Gradle, etc. every single run. I've tried different combinations of volume destinations as well. – Morsel

@NeelKamath "running commands in the Dockerfile" is part of " building Docker images"! And BuildKit is made for caching build and accelerating docker builds. Give it a try. – Toreutic

Using BuildKit alone won't solve this issue: by copying the entire context at the start of the build and using RUN, BuildKit will always re-build everything on every code change (because the context changed), but in addition with @Evgeniy Khyst answer it may move toward a better result – Huynh

@PierreB. OK. So any solution will be more complex than I thought. – Toreutic

Answer 4

As the other answers have mentioned, docker caches each step in a layer. If you could somehow get only the downloaded dependencies into a layer, then it would not have to be re downloaded each time, assuming the dependencies haven't changed.

Unfortunately, gradle doesn't have a built-in task to do this. But you can still work around it. Here's what I did:

# Only copy dependency-related files
COPY build.gradle gradle.properties settings.gradle /app/

# Only download dependencies
# Eat the expected build failure since no source code has been copied yet
RUN gradle clean build --no-daemon > /dev/null 2>&1 || true

# Copy all files
COPY ./ /app/

# Do the actual build
RUN gradle clean build --no-daemon

Also, make sure your .dockerignore file has at least these items, so that they're not sent in the docker build context when the image is built:

.gradle/
bin/
build/
gradle/

Horvitz

Answer 5

I used a slightly different idea. I scheduled a nightly build on my Jenkins building the entire Gradle project:

docker build -f Dockerfile.cache --tag=gradle-cache:latest .

# GRADLE BUILD CACHE
FROM gradle:6.7.1-jdk11

COPY build.gradle.kts /home/gradle/code/
COPY settings.gradle.kts /home/gradle/code/
COPY gradle.properties /home/gradle/code/
COPY ./src /home/gradle/code/src

WORKDIR /home/gradle/code

RUN gradle bootJar -i -s

Then I start my builds from this "cache image" so I can leverage all the Gradle goodness:

docker build --tag=my-app:$version .

# GRADLE BUILD
FROM gradle-cache:latest as gradle

COPY build.gradle.kts /home/gradle/code/
COPY settings.gradle.kts /home/gradle/code/
COPY gradle.properties /home/gradle/code/

RUN rm -rf /home/gradle/code/src
COPY ./src /home/gradle/code/src

WORKDIR /home/gradle/code

RUN gradle bootJar -i -s

# SPRING BOOT
FROM openjdk:11.0.9.1-jre

COPY --from=gradle /home/gradle/code/build/libs/app.jar app.jar

EXPOSE 8080
ENTRYPOINT ["java", "-Xmx2G", "-Djava.security.egd=file:/dev/./urandom", "-jar", "app.jar"]

Remember about pruning unused images every week or so.

Kamacite

Answer 6

Just as an addition to other people answers, if your internet connection is slow, as it downloads dependencies every single time, you might want to set up sonatype nexus, in order to keep the dependencies already downloaded.

Necessitate

Answer 7

I don't know much about docker internals, but I think that the problem is that each new docker build command, will copy all files and build them (if it detects changes in at least one file). Then this will most likely change several jars and the second steps needs to run too.

My suggestion is to build on the terminal (outside of docker) and only docker build the app image.

This can even be automated with a gradle plugin:

• https://github.com/Transmode/gradle-docker (one example, I did not search thoroughly)

Vhf

So gradle building in docker is a wrong way to go? The idea was that you won't need any dependencies installed to build and run code in your enviroment. – Mcgaw

Oh I see! I don't think you mention that on your question. In that case it looks like the current solution is fine... it will take time. Another question is, why do you want your dev env not to have the dependencies? it is called a dev env because it will have dev stuff in it. – Vhf

That's a good point. I should be more specific. All that docker in container development was caused by the fact that project is being edited by like 10 people. So I tought I will be nice to do not have any OS or sdk dependencies. But maybe that is an overkill. – Mcgaw

In my experience (teams up to 6/7 devs) everyone has the local setup. Usually there is a readme file on the each repo root with the steps commands and all that needs setup, for that repository. I understand your problem, but I don't think that docker is the right tool for this. Maybe, try to simplify / minize the setup needed in the first place, for example: either by refactor code, setting better defaults, using naming conventions, less dependencies, better readme setup docs. – Vhf

I feel strongly that building entirely within docker build command is a pretty normal thing to do, so much that I would not have expected them to have had to explain that – Juvenile