can we use dynamic jobs names in gitlab-ci.yml?

Question

I have to migrate from jenkins to gitlab and I would like to be able to use dynamic job names in order to have some information directly in the pipeline summary without having to click on each job etc.... in jenkins we can immediately see the parameters passed to our job and this is not the case in gitlab-ci.

My test runner being on a windows I tried to define the yml as follows:

job_%myParam%:
  stage: build
  script:
    - set>varList.txt
  artifacts:
    paths:
    - varList.txt

When I start my job with %myParam%=true, the variable is not interpreted in the job name and so it takes the name of job_%myParam% instead of expected "job_true".

Is that even possible?

thks :)

Skilling

Coming soon feature: gitlab.com/gitlab-org/gitlab/-/issues/23672 – Rai

@Rai The scope of the issue you linked is a not about dynamic job names per se. It sets out with the issue of job names requiring to be unique across the pipeline and suggests to effectively introduce a job "display name" property, which i agree may solve the problem at hand nevertheless. What indication do you have that it will be implemented anytime soon though? – Salchunas

@Philzen: just a guess from its activity log. Many years so far. So, I might be wrong. – Rai

Answer 1

As of gitlab 12.9, this can be done using trigger and child pipelines — although a little involved:

Quoting the example from the gitlab doc:

generate-config:
  stage: build
  script: generate-ci-config > generated-config.yml
  artifacts:
    paths:
      - generated-config.yml

child-pipeline:
  stage: test
  trigger:
    include:
      - artifact: generated-config.yml
        job: generate-config

In your case, you would put the definition of job_%myParam% in job.yml.in, then have the script generate-ci-config be e.g. sed "s/%myParam%/$PARAM/g" job.yml.in.

This is probably a bit much for just changing the names of course, and there will be a cost associated to the additional stage; but it does answer the question, and may be useful to do more, like start with different versions of the parameter in the same pipeline.

Orcinol

Child pipelines limit you in how nested you can go – Swampy

in gitlab 14.3 there is also "variable inside variable" enabled gitlab.com/gitlab-org/gitlab-runner/-/issues/1809 – Diva

AFAICT this approach should also allow defining Gitlab jobs completely dynamically during runtime, i.e. not just their names but also what they do. Very nice! – Pyrrho

Answer 2

This is now possible via Job Input Parameters which were added in Gitlab 15.11

To achieve your desired result – e.g. have your job be dynamically named job_fourtytwo in a specific run:

1. define your job like this:

   # jobs/dump_env.yaml
   
   "job_$[[ inputs.myParam ]]":
     stage: build
     script:
     - set>varList.txt
     artifacts:
       paths:
       - varList.txt
   

2. set myParam when including the job:

   # e.g. ./.gitlab-ci.yml
   
   include:
   - local: jobs/dump_env.yml
     inputs:
       myParam: fourtytwo
   

---

You can also define a description and default value for your inputs, which will be assumed in case the input is not provided in an include:

# jobs/dump_env.yaml

spec:
  inputs:
    myParam:
      description: My awesome custom job prefix
      default: fourtytwo
---
"job_$[[ inputs.myParam ]]":
# …

Refer to the Define input parameters to includable CI/CD configuration files blog post or the above linked gitlab docs reference for more examples.

Salchunas

Answer 3

No, it is not possible to have dynamic job names in GitLab CI.

Cursed

Since gitlab 15.11 it absolutely is, see https://mcmap.net/q/635542/-can-we-use-dynamic-jobs-names-in-gitlab-ci-yml – Salchunas

Answer 4

Here is a more specific example of how dynamic job can be generated:

generate-job:
  stage: .pre
  script:
    - |
      cat > generated-job.yml <<- EOM
      job_${myParam}:
        stage: build
        script:
          - echo \$CI_COMMIT_REF_NAME
        artifacts:
          paths:
          - varList.txt
      EOM
      cat generated-job.yml
  artifacts:
    paths:
      - generated-job.yml

run-job:
  stage: build
  trigger:
    include:
      - artifact: generated-job.yml
        job: generate-job

Feld