Cannot connect to the daemon docker in a docker image of docker

Question

I am trying to build CI with gitlab, I go from a docker image of docker, and i didn't have any problem with my front repository, but now with the back withe the same gitlab-ci configuration file, i have this daemon error.

Here is the output of the build :

[0KRunning with gitlab-ci-multi-runner 1.10.4 (b32125f)[0;m
[0;m[0KUsing Docker executor with image docker:1.13.1 ...
[0;m[0KPulling docker image docker:1.13.1 ...
[0;mRunning on runner-4e4528ca-project-1649638-concurrent-0 via runner-4e4528ca-machine-1487688057-7c0f1e46-digital-ocean-4gb...
[32;1mCloning repository...[0;m
Cloning into '/builds/***/formation-back'...
[32;1mChecking out af7cbcae as docker...[0;m
[32;1mSkipping Git submodules setup[0;m
[32;1m$ docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com[0;m
Warning: failed to get default registry endpoint from daemon (Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?). Using system default: https://index.docker.io/v1/
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
[31;1mERROR: Build failed: exit code 1
[0;m

Here is my .gitlab-ci.yml :

image: docker:1.13.1

stages:
  - build
  - test
  - deploy

variables:
  BUILD_IMG: $CI_REGISTRY_IMAGE:$CI_BUILD_REF
  TEST_IMG: $CI_REGISTRY_IMAGE:$CI_BUILD_REF_NAME
  RELEASE_IMG: $CI_REGISTRY_IMAGE:latest
  AWS_STAGING_ENV: "***"
  AWS_PROD_ENV: "***"
  DOCKERRUN: Dockerrun.aws.json
  DEPLOY_ARCHIVE: ${AWS_APP}-${CI_BUILD_REF}.zip

before_script:
  - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
  - .ci/before_script

build:
  stage: build
  script:
  - docker build --pull -t $BUILD_IMG .
  - docker push $BUILD_IMG

test:
  stage: test
  script:
  - docker pull $BUILD_IMG
  - docker run --rm $BUILD_IMG npm run test
  - docker tag $BUILD_IMG $TEST_IMG
  - docker push $TEST_IMG

deploy:staging:
  stage: deploy
  environment: Staging
  variables:
    DOCKER_IMG: ${CI_REGISTRY_IMAGE}:${CI_BUILD_REF}
  script:
  - ./.ci/create-deploy-archive $DOCKER_IMG $AWS_BUCKET $DOCKERRUN $DEPLOY_ARCHIVE
  - ./.ci/aws-deploy $DEPLOY_ARCHIVE $CI_BUILD_REF $AWS_STAGING_ENV
  artifacts:
    paths:
    - $DEPLOY_ARCHIVE
  except:
  - production

deploy:production:
  stage: deploy
  environment: Production
  variables:
    DOCKER_IMG: ${CI_REGISTRY_IMAGE}:latest
  script:
  - .ci/push-new-image $TEST_IMG $RELEASE_IMG
  - .ci/create-deploy-archive $DOCKER_IMG $AWS_BUCKET $DOCKERRUN $DEPLOY_ARCHIVE
  - .ci/aws-deploy $DEPLOY_ARCHIVE $CI_BUILD_REF $AWS_PROD_ENV
  artifacts:
    paths:
    - $DEPLOY_ARCHIVE
  only:
  - production
  when: manual

Here is my config.toml file :

concurrent = 1
check_interval = 0

[[runners]]
  name = "***"
  url = "https://gitlab.com/ci"
  token = "750c63cba1c269d789bdb33c42b726"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "alpine:3.5"
    privileged = true
    disable_cache = false
    volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
  [runners.cache]

Here is docker info :

DEBU[0771] Calling GET /v1.24/info
Containers: 1
 Running: 1
 Paused: 0
 Stopped: 0
Images: 1
Server Version: 1.12.6
Storage Driver: devicemapper
 Pool Name: docker-202:1-395267-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 10.74 GB
 Backing Filesystem: xfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 519 MB
 Data Space Total: 107.4 GB
 Data Space Available: 6.569 GB
 Metadata Space Used: 1.397 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.146 GB
 Thin Pool Minimum Free Space: 10.74 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 WARNING: Usage of loopback devices is strongly discouraged for production use. Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.93-RHEL7 (2015-01-28)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge overlay null host
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options:
Kernel Version: 4.4.44-39.55.amzn1.x86_64
Operating System: Amazon Linux AMI 2016.09
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 995.2 MiB
Name: ip-172-31-30-143
ID: D6DU:OBWL:R3HK:DSZK:EOYC:5EHS:NU4I:4M3T:H5PL:JWLH:CIPD:I7VW
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 20
 Goroutines: 27
 System Time: 2017-02-22T11:16:19.042666914Z
 EventsListeners: 0
Registry: https://index.docker.io/v1/
Insecure Registries:
 127.0.0.0/8

Monia

Could you add the result of docker info from the machine hosting gitlab? – Radman

Could you add the content of the config/config.toml file of your gitlab-runner? – Ass

I have the same problem. I set privileged = true in config.toml. I also run into it when starting the docker image manually. – Exhaust

I edited with the config.toml and the docker info – Monia

Answer 1

You need to add

services:
  - docker:dind

to your .gitlab-ci.yml. This tells the runner to start a second container (docker:dind), which is an image of a working docker daemon. It needs to be in a second image in order to run.

For more Information, see the docker example project: https://gitlab.com/gitlab-examples/docker/blob/master/.gitlab-ci.yml

It doesn't work without the service because there is no running docker daemon and you can't run your build inside the docker:dind container because the run command would replace the docker daemon.

Exhaust

Hello Josef Hoppe thank you :) I didn't kwo this service, I solved my problem running two runners for my two projects. I don't know what it worked that way :/ – Monia

Answer 2

What worked for me was to disable TLS by adding this in your runners section environment, as per https://about.gitlab.com/releases/2019/07/31/docker-in-docker-with-docker-19-dot-03/

variables:
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: ""

Hyonhyoscine

Answer 3

The image docker:dind failed, but docker:18.09.7-dind worked for me.

services:
  - name: docker:18.09.7-dind

Here is Gitlab issue details: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2623#note_206835782

Conjoin

Answer 4

Based on this issue https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27215 you will find a working solution. The key is using the healthcheck on docker-in-docker, it waits until the service is actually ready before using it. I encountered this on new nodes in the cluster which randomly failed.

Here is an example using the Kubernetes executor without tls:

Runner config:

    config: |
      [[runners]]
        name = "whatever"
        url = "{{ .Values.gitlabUrl }}" 
        executor = "kubernetes"
        [runners.kubernetes]
          namespace = "{{.Release.Namespace}}"
          image = "docker:24.0.5"
          privileged = true

Pipeline variables:

variables: 
  DOCKER_HOST: tcp://docker:2375 
  DOCKER_TLS_CERTDIR: ""

Build job:

build-docker:
    image: docker:24.0.5
    stage: build
    services:
      - name: docker:24.0.5-dind
        command: [ "--tls=false" ]
        variables:
          HEALTHCHECK_TCP_PORT: "2375"
    script:
      - docker build -t whatever .

Deforest