Difference between different GitLab CI Merge Request rules

Question

As several other users, I'm facing issues with duplicate pipelines in GitLab CI/CD. While there is some documentation on how to prevent this scattered around in the GitLab docs, my impression is that the indivdual docs pages and sections are rather inconsistent.

My question is, what are the differences between the following rules? Or, more specifically, are there cases in which these rules are evaluated differently?

• Switch between branch pipelines and merge request pipelines suggest this to identify merge request pipelines:

  if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
  

• Additionally, Switch between branch pipelines and merge request pipelines also uses this rule in combination with $CI_COMMIT_BRANCH &&:

  if: '$CI_OPEN_MERGE_REQUESTS'
  

• Moreover, the MergeRequest-Pipelines.gitlab-ci.yml uses a third rule:

  if: $CI_MERGE_REQUEST_IID
  

Any explanation or hints to docs pages I might have overlooked is highly appreciated.

Limburg

Answer 1

In order to avoid duplicate pipeline creation and the requirement that you want to switch between Branch-Pipelines and Merge-Request-Pipelines I recommend using these workflow rules

workflow:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
      when: never
    - if: '$CI_COMMIT_BRANCH' || '$CI_COMMIT_TAG'

There is another SO-question that asks how to prevent duplicate pipelines here

The Explanation(s):

In the following section I will try to explain your different rules and how GitLab CI will evaluate them during pipeline creation.

The merge_request_event-rule

Using this rule:

if: '$CI_PIPELINE_SOURCE == "merge_request_event"'

will create a pipeline each time a Merge-Request is created/updated, but there will also be a pipeline created for the branch if you do not have another prevention mechanism (rule). As the variable naming also points out, this is about the source of the pipeline trigger, other sources could be schedule, push, trigger etc.

The CI_OPEN_MERGE_REQUESTS variable:

Using a rule like:

if: '$CI_OPEN_MERGE_REQUESTS'

GitLab will create new pipelines if there is an open Merge-Request for this branch. Pipelines because there will be a Merge-Request pipeline (denoted with the detached flag) and a branch pipeline for the branch you pushed changes.

if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'

This rule above will create a pipeline for your branch when, and only when there is an open MR on that branch.

if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
when: never

When using the above combination no pipeline will be created if there are open Merge-Requests on that branch, which might also be undesirable since the CI should run tests for branches and/or Merge-Requests.

But how to be able to have pipelines for MRs and Branches, but prevent duplications in pipeline creation?

- if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
  when: never
- if: '$CI_COMMIT_BRANCH' || '$CI_COMMIT_TAG'

With this rule set above GitLab will create pipelines for branches and Merge-Requests (the detached ones), as well as pipelines for git-tags, but it will prevent GitLab from duplicating pipelines. The last rule evaluates to true either when there is a commit to a branch or there is a git-tag.

Further links

• Official docs on switching between MR- and Branch-Pipelines
• Docs on how to avoid duplicate pipelines with rules examples

Gorey

Thanks, I think I know understand the difference between $CI_PIPELINE_SOURCE == "merge_request_event" and $CI_OPEN_MERGE_REQUESTS. One point your answer left open is the $CI_MERGE_REQUEST_IID variable, used in the official workflow for MR pipelines. Do you know what is the difference between $CI_MERGE_REQUEST_IID and $CI_OPEN_MERGE_REQUESTS? – Unprecedented

For our workflow, we're using the following rules: yaml workflow: rules: - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' - if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS && $CI_PIPELINE_SOURCE == "push"' when: never - when: always This way, we expect to also run scheduled or manual pipelines. Could there be anything wrong with this configuration? – Unprecedented

Using the rule if: $CI_MERGE_REQUEST_IID is basically the same as a non-nullish check of the variable. This means, if the variable is not null (which would be the case when no MR is present for the current branch) then a pipeline will be created. Your workflow is valid way to handle job-limiting but I think there might be cases where your workflow might create duplicate pipelines, e.g. a tagged commit being pushed to a branch with no open MR. But it might be that I am wrong with that assumption – Gorey

Hello @SPMSE, With the above workflow rules, how to avoid the job not to run on Main branch? After a MR from branch to Main. Still that job get executed.? – Kinsman

@Kinsman you would need to add a new rule, like so: - if: $CI_COMMIT_SOURCE == $CI_DEFAULT_BRANCH when: never This rule should be added to the respective job as first rule within the rules list. – Gorey

Answer 2

I daresay the accepted reply is an antipattern. To get a deterministic result - prefer filtering by the event-source whenever such a source is supported.

ATM, the following sources appear in the docs: push, web, schedule, api, external, chat, webide, merge_request_event, external_pull_request_event, parent_pipeline, trigger, or pipeline.

I also add env-vars to help jobs in the pipe recognize the mode without repeating that logic. These variables propagate to child pipelines, making context recognition consistent and simpler.

e.g.:

workflow:
  rules:
    - # new commits to open merge requests
      if: $CI_PIPELINE_SOURCE == "merge_request_event"
      variables:
        PIPE_MODE: MR #i.e - merge request
    - # manual triggering
      if: $CI_PIPELINE_SOURCE == "trigger"
      variables:
        PIPE_MODE: manual
    - # pushed tags
      if: >
        $CI_PIPELINE_SOURCE == "push" &&
        $CI_COMMIT_TAG
      variables:
        PIPE_MODE: GIT_TAG
    - # all pushed commits to the main branch 
      #   TRICKY! mind that tags were checked before in the priority
      # in most real-world pipelines I saw this clause has `when: never` instead of `variables`, 
      # but since you asked about it - I assume you have a use-case for it
      if: >
        $CI_PIPELINE_SOURCE == "push" &&
        $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      variables:
        PIPE_MODE: RC  #i.e - release-candidate

Then, down the pipeline and child pipelines - context-dependent jobs can be labeled with no husstle like:

if: "RC" == $PIPE_MODE

If you need the job to run in multiple cases - here's the trick:

if: ",RC,TAG," =~ ",$PIPE_MODE," .

Mind how the commas before and after in both sides of the pattern matcher make the match work no matter where in the list the searched mode appears.

If your logic grows to end up with many weird mode names - this trick will help you ensure only exact matches, (as opposed to a long mode name that contains a shorter mode name).

This is obviously a work around flaws in the supported source resolution, but let's not throw the baby with the water...

Frankhouse