Clearing the pipeline cache with Gitlab CI

Question

Is it possible to invalidate or clear a pipeline cache with the Gitlab CI after a pipeline completes?

My .gitlab-ci.yml file has the following global cache definition

cache:
  key: "%CI_PIPELINE_ID%"
  paths:
    - './msvc/Project1`/bin/Debug'
    - './msvc/Project2`/bin/Debug'
    - './msvc/Project3`/bin/Debug'

The cache-key value specifies that each pipeline should maintain it's own cache, which is working fine, but the cache file continues to exist after the pipeline completes. With hundreds of pipelines being run, the size starts to add up and manually deleting the cache folder on our machine isn't a great solution.

I tried adding a cleanup job at the end of the pipeline

cleanup:
  stage: cleanup
  script:
  - rm -rf './msvc/Project1/bin'
  - rm -rf './msvc/Project2/bin'
  - rm -rf './msvc/Project3/bin'
  when: always

Which deletes the local files, but won't delete them from the cache.

Am I missing something here?

Currently running Gitlab-EE 10.3.3

Prognostication

in your case I would rather use artifacts than cache. artifacts can be passed from one job to the other within a pipeline and they are not persistent between pipelines. you can deactivate cache alltogether – Tachograph

Yes, you are absolutely correct. That's the conclusion we finally came to as well. – Prognostication

Answer 1

It's not a perfect solution, but we ended up creating a cleanup job at the end of our .gitlab-ci.yaml file that deletes the cache directory from the filesystem.

This way, each pipeline gets its own unique cache, without cluttering up the file system over time.

cleanup_job:
  stage: cleanup
  script:
    - echo "Cleaning up"
    - rm -rf "%CACHE_PATH%/%CI_PIPELINE_ID%"
  when: always

where

CACHE_PATH: "C:/gitlab-runner/cache/group/project/repo/"

Prognostication

There's also the after_script directive – Mousey

@Mousey yea, but that runs after EACH job. there would be no reason to cache if you were cleaning up the cache after each job – Nervine

This seems not working with docker runs. Seems like the path is different for each kind of build. It would be nice to have an option like the cleanWS in Jenkins. – Hamiltonian

Answer 2

Artifacts are the solution as mentioned in the comments. However there is an option to clear caches in the Pipelines page as shown in the image below.

Mania

Not sure, if this was removed. However, at current non-commercial gitlab the button does not exist. – Covert

I can still find it in my projects although I am using the free version. I can find it even in new projects that I create. – Mania

Fissehay You need to be maintainer to clear the cache. Thats annoying, because a developer can push to .gitlab-ci.yml doing the same. – Covert

Note that the Clear Runner Caches does not actually delete the cache data. You have to delete it manually – Lehrer

Answer 3

Quote from @ayufan one of the project masters of Gitlab:

The cache is stored in /home/gitlab-runner/cache or as docker container docker ps | grep -cache-.

There fore you can try to delete the directory and purge all returned docker containers.

Arnelle

I'm aware where the caches are stored. My goal is to avoid having to manually go and periodically delete them from our build server. – Prognostication

If it is the manual task which is bothering you, you can use Cronjobs or Scheduled Pipelines. But directly deleting cache within the same runtime is not possible at the moment. – Arnelle

The problem with deleting it manually is that I don't know from terminal which pipeline cache should be cleared or persisted. It seems to me that the cache should detect if a file has been removed by the running script, and update the cache – Prognostication

Answer 4

The button "Clear runner caches" mentionned in another answer is not satisfactory:

• it clears the cache for all slugs (all branches, all OSes, etc.), which can be more than what you want
• it does not actually delete the data (as mentioned in the answer comments)

What we ended up doing is create a manually-triggable stage named "clear_cache" that will use the appropriate slug and make the cache empty by removing the usually cached files and directories before generating the cache.

Important: for the top-level directories to cache, you cannot simply remove them, otherwise gitlab-ci will show:

Creating cache main-Win-64-31-protected...
WARNING: some-top-level-directory: no matching files                  
Archive is up to date!                             
Created cache
Cleaning up project directory and file based variables
Job succeeded

and some-top-level-directory is left unchanged in the cache!

so what you need to do is to make them empty instead of removing them. I have not tested what happens for top-level files.

After that you get the following output:

Creating cache cache main-Win-64-31-protected...
some-top-level-directory: found 1 matching files and directories      
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally. 
Created cache
Cleaning up project directory and file based variables
Job succeeded

When debugging caching in gitlab-ci it can be helpful to look what the cache actually contains by browsing it.

Irrigation