Multiple paths with different expiry time in gitlab-ci runners

Question

I'm having issues with parsing a build directory between stages using Gitlab-CI.

Gitlab-CI wipes the created build artifacts between stages which seems weird. I'm trying to store the build artifacts between the build and the test stage, however the build stage also has the build artifact which I want to keep and also the build artifacts which are required to run the next stage.

Is it possible to have multiple expiry times with different paths using the artifacts option?

I have tried the following, which only keeps the second definition of paths (the build/test* paths), and not the first paths (.dmg) declared.

  artifacts:
    paths:
    - build/*.dmg
    expire_in: 1 week
    paths:
    - build/test1
    - build/test2
    - build/test3
    expire_in: 15 mins

I have tried using the caches however can't seem to get that working... Any suggestions would be great appreciated!

Subroutine

Answer 1

According to the docs it doesn't seem possible but I needed to do something similar so as a workaround I did the following.

In the stage of the build that generates all the artifacts I set an expiry of 15 minutes for all the artifacts including the one that should have a different expiration. In your case build/*.dmg. So my artifacts definition for the build section would be like this:

 artifacts:
    paths:
    - build/*.dmg
    - build/test1
    - build/test2
    - build/test3
    expire_in: 15 mins

After that I defined two jobs in next stage of the pipeline. If you do this the two jobs run concurrently. One of the jobs does what you originally intended for the files: build/test1, build/test2 and build/test3. The other job in this stage of the pipeline should have practically nothing in the script section, maybe something like echo "dummy job". But because the dummy job receives the build artifacts from the previous job we can use it to 'change' the expiry from 15 min to a week by simply creating a second artifact with the same file like so:

artifacts:
    paths:
    - build/*.dmg
    expire_in: 1 week

It is a poor workaround because it duplicates the target artifact, since we'll have one with an expiration of 15min and another with an expiration of 1week.

Pronunciamento

Thanks for the answer! Is there any chance you can give more info encapsulating that? So the first job creates a dummy stage, and then the job which does the build only has the artifacts? – Subroutine

I've tried to better explain myself I hope I'm more coherent now, English is a second language to me – Pronunciamento

Answer 2

It is not possible on Gitlab (yet). Feature request is open: https://gitlab.com/gitlab-org/gitlab/-/issues/18744

Rob Paisley / Clive Makamara answer is a good workaround, we also used that.

Demarche

Answer 3

I would also need this feature.

The workaround proposed by Clive is also the easy one that came to my mind.

Another one would be to extend the build job by storing the wanted artifact into an external place, e.g. to push it in the package registry of the local repo or some cloud storage.

This adds more complexity since the test jobs would also need more logic to retrieve them.

Tablet