Azure Pipelines using YAML for multiple environments (stages) with different variable values but no YAML duplication

Question

Let's suppose I have 3 environments on Azure: Dev, Test and Prod. I have the same pipeline for building and deploying the resources and the code for each one of the environments except for two differences:

• different trigger branch
• different variable values

What is the correct approach for this scenario? Because at least 3 come to my mind, none of which is perfect:

Option 1: I guess I could create a single pipeline on Azure DevOps (triggered by any of 3 branches) with 3 stages for each environment and for each stage add a condition to run depending on the source branch, like this:

condition: eq(variables['Build.SourceBranch'], 'refs/heads/a-branch-name')

and in each stage reference different variables. But this would introduce code duplication in each stage - when adding or modifying a step I would have to remember to edit 3 stages - not desirable.

Option 2: Create 3 separate YAML files in my repository, each one of them with specified trigger branch and referencing the same variable names, then create 3 different pipeline on Azure DevOps, each one of them with different variable values. But this would also introduce code duplication.

Option 3: Create 1 build-and-deploy.yaml file as a template with the steps defined in it and then create another 3 YAML files referring to that template, each with different trigger branch and with different variable values in each Azure Pipeline, like this:

trigger:
  branches:
    include:
    - a-branch-name

steps:
- template: build-and-deploy.yaml
  parameters:
      parameterName1: $(parameterValue1)
      parameterName2: $(parameterValue2)

This seems to be the best option but I haven't seen it used anywhere in the examples so maybe I'm just unaware of downsides of it, if there are any.

Landowner

Answer 1

Here's how to do it with a shared pipeline config that gets included into env-specific pipelines.

To support 2 environments (dev and prod) you'd need:

• 1 shared pipeline yaml
• 2 env-specific yamls, one for each env
• 2 pipelines created in Azure DevOps, one for each env; each pipeline referencing corresponding yaml

pipeline-shared.yml:

variables:
  ARTIFACT_NAME: ApiBuild
  NPM_CACHE_FOLDER: $(Pipeline.Workspace)/.npm

stages:
  - stage: Build
    displayName: Build
    pool:
      vmImage: 'ubuntu-latest'
      demands: npm
    jobs:
      ...

  - stage: Release
    displayName: Release
    dependsOn: Build
    pool:
      vmImage: 'ubuntu-latest'
    jobs:
      ...

pipeline-dev.yml:

# Trigger builds on commits to branches
trigger:
  - dev

# Do not trigger builds on PRs
pr: none

extends:
  template: pipeline-shared.yml

pipeline-prod.yml

trigger:
  - master

pr: none

extends:
  template: pipeline-shared.yml

Lunn

As far as I can tell, this does not use a pipeline "Library" (or variable group), or a pipeline "Environment" or a "Deployment Stage". I'm reading all the Azure DevOps doco trying to understand how these are all meant to be linked up, particularly given that there is a lot of emphasis on moving away from classic and into YAML. This solution does not appear to use any of those things - can you confirm? – Tessellated

This answers a big question for me - stages or pipelines. It looks like you can't have different values for variables without having different pipelines? That means to have a deployments script that takes a different environment for deployment, you need a different pipeline? – Sporadic

Answer 2

According to your description, if you want different stages to share the same repo resource, but their trigger branch and variable values are different.

Regarding trigger branch, you can use expression {{if ......}} to determine the trigger branch condition.

Regarding variable values, you can define templates and variable groups to specify them through parameters.

Here is an example, you can refer to it：

1. First go to Library under Pipelines, click on the Variable group to add a variable group. You can add multiple variables to this variable group.

2. Repo structure:

3. azure-pipelines.yml：

   sample：

    stages:
    - template: stage/test.yml
      parameters:
       ${{if contains(variables['Build.SourceBranch'], 'master')}}:
         variableGroup: devGroup
         stageName: Dev
         test: a
       ${{if contains(variables['Build.SourceBranch'], 'test')}}:
         stageName: test
         test: b
   

stage/test. yml：

parameters:
- name: stageName
  displayName: Test
  type: string
  default: test
  
- name: test
  displayName: Test
  type: string
  default: test

- name: variableGroup
  displayName: Test
  type: string
  default: test

stages:
- stage: Test_${{ parameters.stageName }}
  variables: 
    - group: ${{parameters.variableGroup}}
  jobs:
  - job: Test1
    pool:
      vmImage: vs2017-win2016
    steps:
    - script: echo "Hello Test1"
    - script: echo  ${{ parameters.test }}
    - script: echo $(dev1)

Of course, if you want to use a single variable, you can define the variable directly in yaml without adding a variable group.

Thud

Yes - but I have things like a different CDN ID for each environments, and more importantly different cloud users with different secrets that need to be passed in as secret variables... seems like you can't easily do this with only different stages? – Sporadic

Answer 3

I've been searching for something different, but came across this question of yours. I hope you deployement went well.

I am working on something similar and I implemented option 1. The way to reduce code duplication is to use template files.

I created one template file, and now I am calling it for multiple dev, stage, prod stages. I don't need to change it anywhere other than the template file. I only pass the variables in the main.yml file.

Rosenbaum