Is there a way to use OR condition with needs in GitLab CI job

Question

I am trying to create a job dependency with "OR" condition for previous stage jobs using "needs" in ".gitlab.ci.yml" file but unable to find a solution for this.

.gitlab-ci.yml file ->

stages:
  - build
  - test
  - deploy


Build_job:      
  stage: build
  script:
    - echo "hello from build job"

Test_job1:
  stage: test
  script:
    - echo "Start test 1"
  when: manual

Test_job2:
  stage: test
  script:
    - echo "Start test 2"
  when: manual

Deploy_job:
  stage: deploy
  script:
    - echo "Start deploying the job"
  when: manual
    needs:
      - job: Test_job1
        optional: true
      - job: Test_job2
        optional: true

My aim is either of Test_job1 or Test_job2 is passed Deploy_job should be enabled. But with the above code, I am unable to do so as Deploy_job is getting enabled only when both previous two test jobs are passed.

Is there a way if something can be used like needs: [Test_job1 or Test_job2]?

Giorgio

I think this should work out by adding: allow_failure: true to your Test_job definition. But you need to be careful if you plan on extending the configuration in a way that includes the usage of rules keyword, as this might interfere with the job-level definition of when – Raja

Answer 1

The needs:optional: keyword is confusingly named. It is for setting up needs: requirements when jobs might not exist in the pipeline due to other pipeline/job logic (the jobs can optionally exist). However, any optional needs:job: names that do exist in the pipeline will all be required.

I see a few options to get close to your desired pipeline:

automation path

You could use the rules: keyword in your Test_job jobs to conditionally add them to the pipeline.

The humans that are deciding which Test_job to run (an assumption based on the fact that you're using when: manual) are likely following decision logic of some sort--or a preset decision logic can be decided on by the team. You would then build that logic into your needs:if: condition to determine whether or not your Test_job# jobs exist in the pipeline--then only the Test_jobs that are added to the pipeline will be required by the Deploy_job:.

For example, if one test should run on your default branch and the other on feature branches, you could use the following to automatically run the relevant Test_job and have the Deploy_job depend on it:

Test_job1:
  stage: test
  script:
    - echo "Start test 1"
  # this job is only added for pipelines on the default branch
  rules:
    - if $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH
      when: on_success
    - when: never

Test_job2:
  stage: test
  script:
    - echo "Start test 2"
  # this job is only added for pipelines NOT on the default branch
  rules:
    - if $CI_COMMIT_REF_NAME != $CI_DEFAULT_BRANCH
      when: on_success
    - when: never

Deploy_job:
  stage: deploy
  script:
    - echo "Start deploying the job"
  # this is the default value, it's added here to show the automation difference from the question's pipeline
  rules:
    - when: on_success
  # depend on Test_job1 AND Test_job2 if they individually exist in the pipeline
  needs:
    - job: Test_job1
      optional: true
    - job: Test_job2
      optional: true

manual path

You could swap when:manual back in place for when:on_success if you still need humans in the loop on triggering the test jobs or deploy job.

really manual path

Alternatively, if you really just want your humans to manually make the Test_job: decisions ad-hoc, then just remove the needs: keyword from your Deploy_job: and use the existing Stage dependency structure. The when: manual key implicitly sets allow_failure: true, so your jobs are already optional as is. If your humans can't be trusted to manage the "Test before Deploy" dependency on their own with manual jobs, then take the humans out of the loop and see the "automation path" above.

Epanodos