Is there any way to use a variable for a service connection to azure?

Question

I have been writing some terraform and using Azure Devops to deploy the pipeline. However if I use a variable $(serviceconnection) for the service connection it fails with the following error:

There was a resource authorization issue: "The pipeline is not valid. Job DeployDev: Step TerraformCLI1 input backendServiceArm references service connection $(serviceconnection) which could not be found. The service connection does not exist or has not been authorized for use. I Have tried authorising it but no luck. Is there any workaround?

The task is a YAML task to use terraform as below :

- task: charleszipp.azure-pipelines-tasks-terraform.azure-pipelines-tasks-terraform-cli.TerraformCLI@0
   displayName: 'Terraform Init'
   inputs:
     command: init
     workingDirectory: $(Agent.BuildDirectory)/a/azuredirectory/terraform
     backendType: azurerm
     backendServiceArm: $(serviceconnection)
     backendAzureRmResourceGroupName: $(ResourceGroupName)
     backendAzureRmStorageAccountName: $(StorageAccountName)
     backendAzureRmContainerName: $(ContainerName)
     backendAzureRmKey: $(AzureRmKey)

Lightish

Hi, any update, could mine or Thomas's answer answer your question? – Shrewmouse

Answer 1

You need to use a Template expression syntax for the service connection variable:

backendServiceArm: ${{ variables.serviceconnection }}

I imagine it's because the service connection needs to be known before the pipeline runs.

Sample use case. Using a variable file called variable.dev.yaml:

variables:
  serviceconnection: my-dev-service-connection-name
...

You could then reference that in your pipeline:

jobs:
- job: myJob
  ...
  variables:
  - template: ./variable.dev.yaml
  steps:
  - task: AzureCLI@2
    inputs:
      azureSubscription: ${{ variables.serviceconnection  }}
...

Helman

Answer 2

If you want to use runtime variable like $(serviceconnection), it is not supported now.

You can use ${{ variables.serviceconnection }} as Thomas recommended. But this practice means that you have to specify variables in advance(Before you run the pipeline).

For service connections, you can specify a value directly or use the ’compile-time variable‘ ${{xxx}}, which will expand and then populate the service connection section with values before running. In this usage of $(xxx), the service connection of the task cannot be obtained, because this is a runtime value.

The service connection needs to be specified before running. The changes (runtime changes) of the variables during the pipeline run will not be acquired by the service connection part of the subsequent task.

You are using a runtime variable.

But run time variables aren't supported for service connection OR azure subscription. The variable will get initialized at the run time.

https://github.com/microsoft/azure-pipelines-tasks/issues/10376#issuecomment-514477023

You can follow below method to use different service connection:

https://mcmap.net/q/875880/-how-to-use-different-service-connection-for-every-stage-in-azure-pipelines

But still need point that, parameters are expanded just before the pipeline runs, hardcode the specific service connection is unavoidable, this is by design.

Also clearly in this official document:

https://learn.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints?view=azure-devops&tabs=yaml#use-a-service-connection

Shrewmouse

This is clearly false. I have been using variable files since 2y and always specify service connection using variable and it s been working great. – Helman

@Helman I understood the original question as to using 'variable', I know what you mentioned. However, due to the characteristics of the service connection, the changes generated at runtime cannot be acquired by the service connection part. I think the '${{ variables.xxx}}' in the service connection part at this time is a kind of constant in this case. If just talking about how to use the values in the 'variables' of the 'pipeline YAML structure', you're right. – Shrewmouse

@Helman But the value of the service connection here is not able to change in the pipeline run (it needs to be provided before starting), I think the things provided here cannot be called 'variable'. If I misunderstand something, please feel free to let me know, thanks.:) – Shrewmouse

The issue with this answer is that it is misleading. People will think it s not possible but 99% of time we just need to re-use a job with a different set of pre-defined variables per environment. – Helman

Answer 3

One workaround working for me.

Create a common variable group (Library) and define one or more serviceconnection names.

Now, this group needs to be declared as root-level or pipeline-level.

name: Azure CI/CD

pr:
  branches:
    include:
      - develop

trigger:
  branches:
    include:
      - develop

pool:
  vmImage: ubuntu-latest

variables:
- group: CommonVariables

stages:
  - stage: Build        
    jobs:
      - job: B1
        steps:
          - task: AzureCLI@2
            displayName: Preview Bicep Changes
            inputs:
              azureSubscription: '$(azureResourceManagerServiceConnection)'
              scriptType: 'bash'
              scriptLocation: 'inlineScript'
              inlineScript: |
                az --version

Shibboleth

Answer 4

One approach that worked for me, is to move the declaration of variables group to the root of the pipeline script.

Before :

pool:
  vmImage: windows-2022

stages:
  - stage: BuildDevelopment
    variables:
      - group: group-development

    steps:
      - task: SonarCloudPrepare@1
        displayName: "Prepare SonarCloud Analysis"
        inputs:
          SonarCloud: "$(sc_endpoint_name)"
          # Error will occur when pipeline tries to access this part

After :

pool:
  vmImage: windows-2022

variables:
  - group: group-development

stages:
  - stage: BuildDevelopment
    steps:
      - task: SonarCloudPrepare@1
        displayName: "Prepare SonarCloud Analysis"
        inputs:
          SonarCloud: "$(sc_endpoint_name)"

Joliejoliet