Can group name variable be dynamic in azure pipelines?

Question

I have two environments on azure. One difference between them is only environment variables that came from variable groups. Is it possible to set up group name dynamically for one pipeline instead of set up two pipelines that can map their own group variables? It is an example of my build pipeline

trigger:
  - master
  - develop


jobs:

- job: DefineVariableGroups
   steps:
    - script: |
      if [ $(Build.SourceBranch) = 'refs/heads/master' ]; then
        echo "##vso[task.setvariable variable=group_name_variable;isOutput=true]beta_group"
      elif [ $(Build.SourceBranch) = 'refs/heads/develop' ]; then
        echo "##vso[task.setvariable variable=group_name_variable;isOutput=true]alpha_group"
      fi
    name: 'DefineVariableGroupsTask'
  - script: echo $(DefineVariableGroupsTask.group_name_variable)
    name: echovar # that works.

- job: Test
  dependsOn: DefineVariableGroups
  pool:
    vmImage: 'Ubuntu-16.04'
  variables:
    - group: $[ dependencies.DefineVariableGroups.outputs['DefineVariableGroupsTask.group_name_variable'] ]
    # that doesn't work. Error here
steps:
  - script: echo $(mode)
    displayName: 'test'

Directive

I don't understand the question. Each environment can have a different variable group associated with it. Is this a build pipeline or a release pipeline? – Tergiversate

Sorry for unclear question. It's a build pipeline. What I want is to have one build pipeline similar to the above one. But I want also to make it run with a specific variable group which can be calculated depends on the branch name – Directive

@alexche8, Could you please share the complete YAML file, I could not reproduce your problem with current YAML, it always throw the error "there were validation errors or warnings,..Mapping values are not allowed in this context." for the line name: 'DefineVariableGroupsTask'? – Hedvig

Answer 1

One approach or workaround that you can use is with templates. I'm sure that the best option is the possibility to use "group variable" dynamically, while it isn't possible I believe that this is one option.

trigger:
  tags:
    include:
    - develop
    - qa
    - production

variables:
- name: defaultVar
  value:  'value define yml on project repo'
- group: variables-example-outside


resources:
  repositories:
    - repository: yml_reference
      type: github
      ref: refs/heads/yml_reference
      name: enamba/azure_devops
      endpoint: enamba

jobs:
- ${{ if eq(variables['Build.SourceBranch'], 'refs/tags/develop') }}: 
  - template: deploy.example.yml@yml_reference
    parameters:
      GroupVariablesName: variables-example-developer

- ${{ if eq(variables['Build.SourceBranch'], 'refs/tags/qa') }}: 
  - template: deploy.example.yml@yml_reference
    parameters:
      GroupVariablesName: variables-example-qa

- ${{ if eq(variables['Build.SourceBranch'], 'refs/tags/production') }}: 
  - template: deploy.example.yml@yml_reference
    parameters:
      GroupVariablesName: variables-example-production

Inside of template:

parameters:
  GroupVariablesName: ''

jobs:
- job: deploy_app
  displayName: 'Deploy application'
  variables:
  - group: ${{ parameters.GroupVariablesName }}
  - group: variables-example-inside

  steps:
    - script: |
        echo outsidevar '$(outsidevar)'
        echo defaultVar '$(defaultVar)'
        echo var by tag '$(variable)'

Apocalypse

Answer 2

Can group name variable be dynamic in azure pipelines?

Sorry for any inconvenience.

I am afraid this is not supported at this moment. So we have to declare the variable group you want to consume in a YAML pipeline.

Some other communities raised the same requirement earlier, and this requirement has been passed to product team, you can check the details from the ticket:

Ticket: Dynamic Variable Groups?

Note: You can vote and add your comments for this feedback. When there are enough communities vote and add comments for this feedback, the product team member will take this feedback seriously.

Hope this helps.

Kremenchug

Thanks, I subscribed on the issue to track it. – Directive

Answer 3

for ones who is looking into it (found here https://www.eliostruyf.com/quick-tip-dynamic-variable-groups-azure-devops-yaml-pipelines/):

variables:
- ${{ if eq(variables['build.SourceBranchName'], 'refs/tags/develop') }}:
  - group: variables_group_master
- ${{ if ne(variables['build.SourceBranchName'], 'refs/tags/master') }}:
  - group: variables_group_dev

Please be careful about the syntax.

Brenna

Answer 4

You can actually use runtime parameters within the pipeline to dynamically reference group names without the need to have them hardcoded in the yaml itself.

A very basic example

parameters:
- name: env
  type: string
  default: dev

variables:
- group: my-group-name-${{ parameters.env }}

so without passing any parameters, the variable group used will be

my-group-name-dev

but perhaps you pass in a parameter to your yaml file, e.g. qa

my-group-name-qa

will be the variable group used.

Donley

Answer 5

There are some Build variables available at compile time. Those that are available in templates are listed in the Microsoft Docs.

There are a few variables over which you have control, for example Build.SourceBranch, or Build.DefinitionName.

Definition name is the name of the pipeline. It can be used to store the name of a variable group and can be accessed at template compile time. A good way of doing this is to use a delimiter to seperate the human readable part of the build definition name from the name of the variable group.

For example for a pipeline called "Update Contoso University #ContUni-w6edf" the following yaml would load a variable group called ContUni-w6edf.

variables:
  - template: 'variables.yml' #some source controlled variables
  - group: "${{ split(variables['Build.DefinitionName'], '#')[1] }}"

Brocatel

Answer 6

I came across a similar situation and thought I would share my results.

While in theory you can utilize group variables in the Pipeline (assuming permissions are set properly) I found out its not the intended workflow for Azure DevOps to load one group over another with some kind of condition in the Pipeline.

So what I did was:

1. For the Pipeline, focus all the tasks to generate artifacts that you will utilize in the Release. Remember it will run for every branch included in the trigger.
2. Then in the Release, choose the artifact and create different stages for each environment, and have those environments linked to the proper variable groups.

This way, the Pipeline can be triggered by commits and run for each individual branch and then do the tasks that require the group variables in the Release portion that gets triggered when the artifact is created. So in conclusion, I get my condition filtering through the different stages in the Release and apply whatever tasks required at that point in the workflow.

I hope this helps someone understand the structure and difference between pipeline and release and save time on structuring your DevOps Build/Release workflow right the first time :)

Nympho

Answer 7

I was running into a similar issue and this worked for me:

for example, you're trying to retrieve a group variable named dev-01

    - name: currentStack
      value: ${{ format('$({0}-{1})', variables['env'], variables['location']) }}

Knockwurst

Answer 8

One of simple solution is to support these different environments per branch would be by creating a variable group per branch/environment.

Now using expressions we can obtain required variable group based on branch name.

  variables:
  - ${{ if eq(variables['build.SourceBranchName'], 'main') }}:
    - group: Prod
  - ${{ if ne(variables['build.SourceBranchName'], 'main') }}:
    - group: DEV

Pelson