Gitlab DOCKER_AUTH_CONFIG not working

Question

The following are in my .gitlab-ci.yml

stages:
  - build

variables:
  DOCKER_HOST: tcp://docker:2375/
  DOCKER_DRIVER: overlay2

services:
  - docker:dind

build-image:
  image: docker:stable
  stage: build
  script:
    - docker build --no-cache -t repo/myimage:$CI_JOB_ID .
    - docker push repo/myimage:$CI_JOB_ID

I've setup the DOCKER_AUTH_CONFIG in Gitlab like following (to contain all possibilities of matching)

{
    "auths": {
        "https://index.docker.io": {
            "auth": "...."
        },
        "https://index.docker.io/v1/": {
            "auth": "..."
        },
        "https://index.docker.io/v2/": {
            "auth": "..."
        },
        "index.docker.io/v1/": {
            "auth": "..."
        },
        "index.docker.io/v2/": {
            "auth": "..."
        },
        "docker.io/repo/myimage": {
            "auth": "..."
        }

    }
}

However, whenever trying to push the image, the following error occurred

$ docker push repo/myimage:$CI_JOB_ID
The push refers to repository [docker.io/repo/myimage]
ce6466f43b11: Preparing
719d45669b35: Preparing
3b10514a95be: Preparing
63dcf81c7ca7: Waiting
3b10514a95be: Waiting
denied: requested access to the resource is denied
ERROR: Job failed: exit code 1

It worked when I use docker login with username/password. Anyone please show me what I did wrong to get it to work with DOCKER_AUTH_CONFIG?

Thanks heaps

Regards Tin

Negrophobe

you can then use docker login command in pipeline too, just make the password as secret and it will be available only in protected pipelines... – Flu

Did you see https://mcmap.net/q/472552/-gitlab-ci-how-to-specify-the-image-that-the-job-runs-on ? – Volotta

Answer 1

To use the content of DOCKER_AUTH_CONFIG as docker login, just store it in $HOME/.docker/config.json, e.g. as follows:

before_script:
  - mkdir -p $HOME/.docker
  - echo $DOCKER_AUTH_CONFIG > $HOME/.docker/config.json

Ref: https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#option-3-use-docker_auth_config

This allows to use a single config to load images for build containers and to access the registry inside the build from the same configuration source.

note: this replaces an execution of docker login

see also: https://docs.docker.com/engine/reference/commandline/login/#privileged-user-requirement

Banda

I got an error when tried your solution: /bin/sh: eval: line 113: can't create /root/.docker/config.json: nonexistent directory – Quietus

ok, in that case, the respective directory does not exist. you can execute mkdir -p $HOME/.docker as first command of the before_script. – Banda

Hey! yeah, that's correct thank you, with that, I could create the docker file and then copy over the docker config. being said that the Auth credentials need to have write permissions to the private repo. but your comment helped me a lot. you are a life saver thank you – Quietus

Answer 2

DOCKER_AUTH_CONFIG works when you are trying to pull the image from your private repository. Here is the function that uses that config variable. That function is only used by getDockerImage function.

So whenever you need to push your image inside your job's script section, you need the docker login step before that.

Fevre

That looks like the link is broken. I am not sure the correct line, however here is the correct file link; gitlab.com/gitlab-org/gitlab-runner/-/blob/main/executors/… – Iconography

Answer 3

The documenation describing DOCKER_AUTH_CONFIG doesn't show any example with several credentials. The documented syntax is:

{
   "auths":{
      "registry.hub.docker.com":{
         "auth":"xxxxxxxxxxxxxxxxxxxxxxxxxxxx" // base 64 encoded username:password
      }
   }
}

Still, as you said, you can use before_script at the beginning of the gitlab-ci.yml file or inside each job if you need several authentifications:

before_script:
  - echo "$CI_REGISTRY_PASSWORD" | docker login -u "$CI_REGISTRY_USER" --password-stdin 

Where $CI_REGISTRY_USER and CI_REGISTRY_PASSWORD would be secret variables.

And after each script or at the beginning of the whole file:

after_script:
    - docker logout

I wrote an answer about using Gitlab CI and Docker to build docker images : How to build, push and pull multiple docker containers with gitlab ci?

Using --password-stdin and secrets instead of a plain -p <password> is a better alternative.

EDIT: The example syntax in mypost is taken from this awesome answer from @Ruwanka Madhushan Can't Access Private MySQL Docker Image From Gitlab CI. You should go see for yourself

SECOND EDIT: You should protect your secret variable only if you want to make them available for protected branches or tags. If you didn't setup any protected brnach or tag, do not use protected variables.

From the doc: Variables could be protected. Whenever a variable is protected, it would only be securely passed to pipelines running on the protected branches or protected tags. The other pipelines would not get any protected variables. https://docs.gitlab.com/ee/ci/variables/#protected-variables

Toggle

It is only one credential but I was not sure which "registry" was correct, so I added all of them. And none of them worked. I didn't open an issue on Github. I ended up using docker login with secret variables setup in GitLab. – Negrophobe

So you want to use your private registry ? Is that it ? Did you try using only one auth without specifying schema ( without http or https) and without URL path ? I would go for a simple { "auths" : { "docker.io": { "auth": "..." } } } – Toggle

Yeah I wanted to use private registry. And I did try with only one single "auth" and it still failed. – Negrophobe

This is what I had { "auths": { "docker.io": { "auth": "..." } } }. And the DOCKER_AUTH_CONFIG is protected. – Negrophobe

Are you sure you want it protected? Are you running tagged branches ? – Toggle

I am not sure what the difference between protected and non protected is. I am not running tagged branches right now. I had this though only: master. I tried it with not protected, still didn't work – Negrophobe

So do not protect the variable. Else it will be accessible only for tagged branches – Toggle

Answer 4

Since I wanted to access my Gitlab container registry from a Gitlab pipeline, I really wanted to use the $CI_JOB_TOKEN. I've achieved this in a relatively clean and clear manner I think.

I've defined the following variables:

variables:
  DOCKER_AUTH: echo -n "gitlab-ci-token:$CI_JOB_TOKEN" | base64
  DOCKER_AUTH_CONFIG: echo {\"auths\":{\"gitlab.container.registry.url\":{\"auth\":\"$(eval $DOCKER_AUTH)\"}}}

And the following before_script which evaluates the above mentioned variables and creates the docker config.json

before_script:
  - mkdir -p $HOME/.docker
  - eval $DOCKER_AUTH_CONFIG > $HOME/.docker/config.json

Dungdungan

Answer 5

In my case the issue was following these docs blindly

https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#define-an-image-from-a-private-container-registry

They tell you to do the following if you need to manually generate the token:

# The use of "-n" - prevents encoding a newline in the password.
echo -n "my_username:my_password" | base64

# Example output to copy
bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=

My password had spaces in so...

# Correct encoding
> echo "username:password with spaces in it" | base64
dXNlcm5hbWU6cGFzc3dvcmQgd2l0aCBzcGFjZXMgaW4gaXQK

# Encoding as in docs
> echo -n "username:password with spaces in it" | base64
dXNlcm5hbWU6cGFzc3dvcmQgd2l0aCBzcGFjZXMgaW4gaXQ=

Discreet

The -n flag is unrelated to spaces in the password. It tells echo to exclude the ending newline. Notice that the encoded result only differs on the last character, which is the newline. If the encoding with -n fails, but the leaving out -n works, it would indicate that your password has an ending newline! The GitLab docs are correct. – Beheld

@Beheld agreed this is what it says in the docs however if you test the commands above you may see different. I guess is may depends on the distro you are using – Discreet

@Beheld agreed this is what it says in the docs however if you test the commands above you may see different. I guess is may depends on the distro you are using – Discreet

Unless the distro has aliased echo to something else (which is unlikely) I don't see how the distro could come into play here. If you use Python to decode your input you can see that it contains an ending newline: python3 -c "import base64; print(repr(base64.b64decode('dXNlcm5hbWU6cGFzc3dvcmQgd2l0aCBzcGFjZXMgaW4gaXQK')))" – Beheld

@Beheld just tested it again on a mac and the results are as in my answer :) – Discreet

Answer 6

If you, like us, have a lot of pipelines and don't want to edit all gitlab ci configs everywhere, you can also configure this once per runner.

In /etc/gitlab-runner/config.toml add a pre_build_script:

[[runners]]
  environment = ["DOCKER_AUTH_CONFIG={\"auths\":{\"https://index.docker.io/v1/\":{\"auth\":\"YOUR TOKEN\"}}}"]
  pre_build_script = "mkdir ~/.docker -p && echo $DOCKER_AUTH_CONFIG > ~/.docker/config.json"

A little more information can be found in Gitlab's docs.

Velamen