Build a Docker image using docker build and a tmpfs?

Question

Question: How do you specify in a Dockerfile, or on the docker build command line, that you would like to have a tmpfs mounted in the build container? This is in the context of a split build - the first container, which would use the RAM disk, builds the application from source, and the second stage copies the result out to a new container.

This question appears to be similar, but my motivations differ. I'm not so much concerned with stale image layers persisting, but I'm concerned with performance of the build. When experimenting outside of Docker, building the particular application I'm working with was over 4x faster when the entire source tree was in RAM rather than on disk. (The project has many intermediate builds and parallelism, so even an SSD tends to thrash around a bit)

Since Docker does support mounting a tmpfs during a normal docker run command, it would seem there should be a way to include this in a Dockerfile? However, I can't seem to find this information anywhere - nearly every search for "tmpfs" and "dockerfile" or "build" or "ramdisk" and so on just points to either the above linked post or Docker's docs on using tmpfs in containers started with docker run.

It would be acceptable if the tmpfs would not persist even through to the second container's assembly. That could be remedied simply by copying the built application out of the tmpfs within the build container before that container exits, and then using that new location when COPYing.

Syrupy

Answer 1

With BuildKit, you can use experimental features to mount a tmpfs filesystem for a single RUN line. This filesystem will not exist outside of the context of that RUN line, just as a tmpfs does not exist when a container has been stopped or deleted, so you'll need to copy any artifacts back into the container filesystem at the end of your build.

For BuildKit, you need at least 18.09, and you can enable it by either:

    export DOCKER_BUILDKIT=1

for a single shell, or to change the default on the host you can update /etc/docker/daemon.json with:

    {
      "features": {"buildkit": true}
    }

With BuildKit enabled, the Dockerfile would look like:

    # syntax=docker/dockerfile:experimental
    FROM your_base_image
    COPY src /src
    RUN --mount=type=tmpfs,target=/build \
        cp -r /src/. /build/ \
     && cd /build \
     && make your_project \
     && cp /build/result.bin /result.bin
    ...

Note that BuildKit is rather new, won't be supported in most cloud build environments, and isn't supported from docker-compose in older versions either. To see more on these experimental features, see: https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/reference.md

Docker-compose:

The support for BuildKit within docker-compose was released in v1.25.0 In order to enable it, do the following configurations:

    export DOCKER_BUILDKIT=1 # or configure in daemon.json
    export COMPOSE_DOCKER_CLI_BUILD=1

With those variables set in your shell, you can now run docker-compose build using BuildKit.

In windows you can execute in your console:

    setx DOCKER_BUILDKIT 1 # or configure in daemon.json
    setx COMPOSE_DOCKER_CLI_BUILD 1

after will need restart your console

Rossetti

btw, we also tested on docker build like: RUN echo 123 > /dev/shm/test.conf, the final image will not include test.conf file. – Urogenital

docker-compose now supports buildkit officially! Refer for the edit suggestion – Gelatinoid

Answer 2

In docker mountpoints are only configurable when you do docker run. Directories in a dockerfile should not be change its behaviour. That's why the mounting points are only in run.

I hope it helps !

Mahaliamahan