Gitlab Docker-in-Docker CI\CD scheme: how to pass-through volumes

Question

Suppose I have repository on Gitlab and following deploying scheme:

1. Setup docker and gitlab-runner with docker executor on host server.
2. In .gitlab-ci.yml setup docker-compose to build and up my service together with dependencies.
3. Setup pipeline to be triggering by pushing commits to production branch.

As it advised in Gitlab guide I've placed in /etc/gitlab-runner/config.toml following lines:

  executor = "docker"
  [runners.docker]
    image = "alpine"
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]

But volumes of my docker-compose.yml are completely ignored. Suppose I have git repository with following structure:

.gitlab-ci.yml
docker-compose.yml
user_conf.d/app.conf

and have volumes: ./user_conf.d:/etc/nginx/user_conf.d. When I check on /etc/nginx/user_conf.d inside the container I find an empty folder intead of folder with app.conf inside.

So the question is: how to properly pass volumes to docker container which is started from docker executer of Gitlab runner.

P.S. Configs are as following:

.gitlab-ci.yml:

image:
  name: docker/compose:latest

services:
  - docker:dind
stages:
  - deploy

deploy:
  stage: deploy
  only:
    - production
  script:
    - docker image prune -f
    - docker-compose build --no-cache
    - docker-compose up -d

docker-compose.yml:

services:
  nginx:
    image: jonasal/nginx-certbot:latest
    restart: unless-stopped
    env_file:
      - ./nginx-certbot.env
    ports:
      - 80:80
      - 443:443
    volumes:
      - /etc/letsencrypt
      - ./user_conf.d:/etc/nginx/user_conf.d

Gastelum

Answer 1

I was facing the same issue on an equivalent setup (self-hosted docker executor, similar .gitlab-ci.yml, volume mounting via Docker compose).

I solved it via the following two changes to section [runners.docker] of config.toml:

1. Flip privileged = false to privileged = true
2. Add "/builds:/builds" to volumes

I got inspiration for both changes from this GitLab issue.

For reference, this is the full config.toml after the changes:

# config.toml

concurrent = 4

[session_server]
  session_timeout = 1800

[[runners]]
  name = "..."
  url = "..."
  token = "..."
  executor = "docker"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
  [runners.docker]
  tls_verify = false
  image = "docker:20.10.17"
  privileged = true
  disable_entrypoint_overwrite = false
  oom_kill_disable = false
  disable_cache = false
  volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock", "/builds:/builds"]
  shm_size = 0

Fiftyfifty