Configure cache on GitLab runner

Question

I don't manage to configure a cache directory, but somehow it does not work. I'm not even sure it takes the config.toml file.

my config.toml:

concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  cache_dir = /tmp/gitlab-runner-cache

... both on /etc/gitlab-runner/config.toml and in ~/.gitlab-runner/config.toml

my .gitlab-ci.yml:

image: docker:latest
services:
  - docker:dind

cache:
  key: "myCache"
  paths:
    - ${CI_PROJECT_DIR]/.m2/

variables:
  DOCKER_DRIVER: overlay
  MAVEN_OPTS: -Dmaven.repo.local=${CI_PROJECT_DIR}/.m2

stages:
  - build
  - acceptance

maven-build:
  image: maven:3.3-jdk-8
  stage: build
  script: "mvn clean package"
  artifacts:
    paths:
      - target/*.jar

maven-acceptance:
  dependencies:
    - maven-build
  image: maven:3.3-jdk-8
  stage: acceptance
  script: "mvn verify"

When I try: gitlab-runner exec shell maven-build

I get the error:

ERROR: Could not create cache adapter               error=cache factory not found: factory for cache adapter "" was not registered

The build is successful, but the .m2 repository is not cached...

Therefore, I have 2 questions:

1. How to know if gitlab-runner actually takes my config.yml?
2. What is a correct config for caching?

Thanks in advance!

Luster

Here is a good thread explaining the cache configuration for maven #37785654 – Canonicity

The error I get is: level=warning msg="Cache config not defined. Skipping cache operation." But the cache does get stored to a local folder. Version: GitLab Community Edition 14.5.2. – Snakemouth

Answer 1

I ran into the same issue just now. I am not sure which version of Gitlab you use, but version 12 supports either an S3-compatible storage or GCS as the cache provider (see thes docs), but no local cache folder like cache_dir = /tmp/gitlab-runner-cache.

The issue I had was, that I forgot to add Type = "s3" to the [runners.cache]section of the Runner configuration:

  [runners.cache]
    Type = "s3"
    Shared = true
    [runners.cache.s3]
      ServerAddress = ...

Dianadiandra

So when I get it right... You need S3 or GCS storage to make a persistent shared/private cloud cache... If you have neither of both, cache is only locally in the container. Correct? So can I delete the "runner.cache" section to avoid the "ERROR: Could not create cache adapter"? – Pantisocracy

Yes, as far as I know. Instead of using AWS or GCS, you may spin up a local Minio S3-compatible server and use it as the S3 cache. This way all caches stay on your server. – Dianadiandra

Answer 2

I had the same error ("cache factory not found") and the problem was that the value of cache_dir was not included in the [runners.docker] / volumes config.

The doc about cache_dir says:

If the docker executor is used, this directory needs to be included in its volumes parameter.

After adding the path to volumes the cache works without S3 or other distributed cache.

Here are the relevant parts of the working /etc/gitlab-runner/config.toml:

[[runners]]
  executor = "docker"
  cache_dir = "/home/gitlab-runner/cache"
  [runners.docker]
    volumes = ["/cache", "/home/gitlab-runner/cache"]

Nynorsk

Actually struggling with this right now - unfortunately, your fix didn't work for me. I'm curious what version of the runner you have? And are you running exec shell or exec docker. Thanks! – Cartel

I'm running gitlab-runner 12.4.1 (Debian package). By exec you mean executor? I'm using docker. – Tryma

@MichaëlWitrant What's the purpose of having both /cache and /home/gitlab-runner/cache defined as volumes? Why not just the latter? – Hyperpyrexia

I'm not sure. I think it was already there so I just kept it and added the missing volume. – Tryma

See gitlab.com/gitlab-org/gitlab-runner/-/issues/… for the correct solution for passing cache around in a local runner between jobs. – Lazaretto