How to use secrets when building docker compose locally

Question

I recently started using Buildkit to hide some env vars, and it worked great in prod by gha!

My Dockerfile now is something like this:

# syntax=docker/dockerfile:1.2
...
RUN --mount=type=secret,id=my_secret,uid=1000 \
    MY_SECRET=$(cat /run/secrets/my_secret) \
    && export MY_SECRET

And my front was something like this:

DOCKER_BUILDKIT=1 docker build \
    --secret  id=my_secret,env="MY_SECRET"

And when I run this on my Github actions, it works perfectly.

But now, the problem here is when I try to build it locally. When performing a docker-compose build it fails. Of course, because I'm not passing in any secret so my backend (Dockerfile) won't be able to read it from run/secrets/.

What I've tried to do, so far, to accomplish the local build using docker-compose build:

1. Working with Docker secrets:

I basically tried doing:

$ docker swarm init
$ echo "my_secret_value" docker secret create my_secret -

I thought that saving a secret would fix the problem but didn't work. I still got the same error message:

cat: can't open '/run/secrets/my_secret': No such file or directory

2. I also tried passing in the secret on my docker-compose file like the following but didn't work either:

version: '3'
services:
  app:
    build:
      context: "."
      args:
        - "MY_SECRET"
  secrets:
      - my_secret
secrets:
  my_secret:
    external: true

3. I also tried storing the secret in a local file, but didn't work, the same error:

version: '3'
services:
  app:
    build:
      context: "."
      args:
        - "MY_SECRET"
  secrets:
      - my_secret
secrets:
  my_secret:
    file: ./my_secret.txt

4. I also tried doing something like this answer something like this:

args:
    - secret=id=my_secret,src=./my_secret.txt

But still got the same error:

cat: can't open '/run/secrets/my_secret': No such file or directory

What am I doing wrong to successfully perform a docker-compose build?

I'm aware that I can easily use two Dockerfiles, a Dockerfile to build in local and a Dockerfile to build in prod but I just want to use Buildkit as it is, by only modifying my docker-compose.yml file.

Does anyone have an idea about what am I missing to be able to build locally reading from /run/secrets/?

Ailssa

Answer 1

Support for this was recently implemented in v2. See the below pull requests.

• https://github.com/docker/compose/pull/9386
• https://github.com/compose-spec/compose-spec/pull/238

The provided example looks like this:

services:
  frontend:
    build: 
      context: .
      secrets:
        - server-certificate
secrets:
  server-certificate:
    file: ./server.cert

So you are close, but you have to add the secret key under the build key.

Also keep in mind that you have to use docker compose instead of docker-compose, in order to use v2 which is built into the docker client.

Oliveira

Hey, I actually also tried this from version 3.8 and isn't working, got the error: services.app.build contains unsupported option: 'secrets' What do I have to do to get it working? – Ailssa

My docker desktop says that my compose version is 1.29.2 – Ailssa

@JonatanLavado, this is a compose v2 feature. Use the build in command docker compose not docker-compose which is the python package. It works for me with Docker Compose version v2.5.0 – Oliveira

I just updated my docker desktop and it seems to be working. I'll let you know. But a quick question, do you know if it's possible to pass an exported variable to the secrets property? I have my secret saved locally with $ export VAR. Is there a property to define it without using a file? – Ailssa

@JonatanLavado, there is currently no support for variables. There is an open issue for this. It may be available in future releases. – Oliveira

Answer 2

I am not sure exactly since which version of compose, but you can pass secrets as environment like this:

version: "3"
services:
  app:
    image: app:latest
    build:
      context: .
      secrets:
        - BUILD_SECRET
secrets:
  BUILD_SECRET:
    environment: BUILD_SECRET

Then handle in Dockerfile like this:

RUN --mount=type=secret,id=BUILD_SECRET BUILD_SECRET="$(cat /run/secrets/BUILD_SECRET)" \
    echo "BUILD_SECRET=$BUILD_SECRET"

Now you just put BUILD_SECRET to .env and docker-compose build will read it by itself.

Anisette