"Could not retrieve credential from local cache for service principal" when using Azure CLI 2.30.0 credentials in Python SDK on Azure Devops MS agent

Question

I an Azure Pipeline on a self-hosted agent I use this task

      - task: AzureCLI@2
        displayName: Azure CLI task with Python SDK
        inputs:
          azureSubscription: 'SUBSCRIPTION-SERVICE-CONNECTION'
          scriptType: bash
          scriptLocation: inlineScript
          inlineScript: |
            python ./magic-script.py

with that I am able to use the credentials to authenticate Azure Python SDK:

client = get_client_from_cli_profile(GraphRbacManagementClient)

When I transfer this process to a MS hosted agent I get this error:

  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/azure/common/client_factory.py", line 85, in get_client_from_cli_profile
    with_tenant=True,
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/azure/common/credentials.py", line 98, in get_azure_cli_credentials
    cred, subscription_id, tenant_id = profile.get_login_credentials(resource=resource)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/azure/cli/core/_profile.py", line 335, in get_login_credentials
    credential = self._create_credential(account, client_id=client_id)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/azure/cli/core/_profile.py", line 592, in _create_credential
    return identity.get_service_principal_credential(username_or_sp_id)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/azure/cli/core/auth/identity.py", line 185, in get_service_principal_credential
    entry = self._msal_secret_store.load_entry(client_id, self.tenant_id)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/azure/cli/core/auth/identity.py", line 270, in load_entry
    .format(sp_id))
knack.util.CLIError: Could not retrieve credential from local cache for service principal ***. Run `az login` for this service principal.

---

Based on this migration guide I also tried AzureCliCredential like

credential = AzureCliCredential()
client = GraphRbacManagementClient(credential, os.environ["subscriptionId"])   

which get's my script signed in - but when using GraphRbacManagementClient I get this error locally on my dev box and on the agent:

    root_group = [g for g in graph_client.groups.list(
  File "C:\Python38\lib\site-packages\msrest\paging.py", line 143, in __next__
    self.advance_page()
  File "C:\Python38\lib\site-packages\msrest\paging.py", line 129, in advance_page
    self._response = self._get_next(self.next_link)
  File "C:\Python38\lib\site-packages\azure\graphrbac\operations\groups_operations.py", line 336, in internal_paging
    response = self._client.send(request, stream=False, **operation_config)
  File "C:\Python38\lib\site-packages\msrest\service_client.py", line 336, in send
    pipeline_response = self.config.pipeline.run(request, **kwargs)
  File "C:\Python38\lib\site-packages\msrest\pipeline\__init__.py", line 197, in run
    return first_node.send(pipeline_request, **kwargs)  # type: ignore
  File "C:\Python38\lib\site-packages\msrest\pipeline\__init__.py", line 150, in send
    response = self.next.send(request, **kwargs)
  File "C:\Python38\lib\site-packages\msrest\pipeline\requests.py", line 65, in send
    self._creds.signed_session(session)
AttributeError: 'AzureCliCredential' object has no attribute 'signed_session'

For StorageManagementClient this works. Seems to be depending on the SDK client used.

Maas

Answer 1

This issue is caused by Azure CLI version 2.30.0 which seemed to be rolled out MS hosted agents recently.

Hence I adapted all my Python scripts running on (MS and self) hosted agents to this model:

def get_graph_client(subscription_id):
    if "tenantId" in os.environ:
        print('using environment variables')
        config_dict = {
            "clientId": os.environ["servicePrincipalId"],
            "clientSecret": os.environ["servicePrincipalKey"],
            "subscriptionId": subscription_id,
            "tenantId": os.environ["tenantId"],
            "activeDirectoryEndpointUrl": "https://login.microsoftonline.com",
            "resourceManagerEndpointUrl": "https://management.azure.com/",
            "activeDirectoryGraphResourceId": "https://graph.windows.net/",
            "managementEndpointUrl": "https://management.core.windows.net/"
        }
        client = get_client_from_json_dict(
            GraphRbacManagementClient, config_dict)
    else:
        print('using CLI credentials')
        credential = AzureCliCredential()
        client = GraphRbacManagementClient(credential, subscription_id)        
    return client

The error AttributeError: 'AzureCliCredential' object has no attribute 'signed_session' I showed in my question was due to corrupt dependencies. Once virtual environment was cleaned up and the correct dependencies where installed, it worked for all Python Azure SDK clients.

This allows me to operate in both modes, with Azure CLI credentials and also with explicit credentials by setting addSpnToEnvironment: true in the corresponding pipeline task:

      - task: AzureCLI@2
        displayName: Azure CLI task with Python SDK
        inputs:
          azureSubscription: 'SUBSCRIPTION-SERVICE-CONNECTION'
          scriptType: bash
          scriptLocation: inlineScript
          addSpnToEnvironment: true
          inlineScript: |
            python ./magic-script.py

Maas

Any ideas why this is failing? Ours started falling over yesterday with this error, although have always been on Hosted agents. Quite annoying if having to pass around authentication to steps - currently most use cli authentication as assume it is already authenticated! – Thrave

Ah, you're saying that worked before? I was not sure because I was just moving some deployments back from self-hosted to hosted agents. – Maas

There was an update to azure CLI recently which stopped generating accessTokens.json. Maybe that's related? github.com/Azure/azure-cli/issues/19707 – Guay

Did not think about that but right. Our agent has version 2.28.0 and MS agent has version 2.30.0. – Maas

I have added an answer for why this seems to happening for us - basically a recent update to azure cli being pushed to the hosted agents which changes the method of authentication. – Thrave

Answer 2

This seems to be due to an update to Azure-CLI between 2.29.0 and 2.30. In our pipeline, the az login task was running in the agent (MS-hosted) directly, so was using that version. Then our interaction with the API was in a conda environment with a pinned, older version of azure-cli. By doing the az login within the same environment as the subsequent calls (bringing the Azure CLI versions in line), that resolved it.

It is detailed in this github issue: https://github.com/Azure/azure-cli/issues/20153#issuecomment-958684723

Thrave