Azure DevOps Pipelines

Question

I am new to working with Azure DevOps, I am trying to create a pipeline using Azure DevOps for deploying my terraform code onto AWS, for authentication I am aware that we can use service principles but that will mean I will need to specify my acess and secret keys in azure DevOps which I do not want to do, so I wanted to check if there are any other ways of doing this?

Yean

you have to authenticate either way, so you'd have to get some sort of credentials into azure devops... – Lava

Answer 1

For accessing/storing these kinds of secrets you can try the Azure Key Vault

Store all your secrets in Azure Key Vault secrets.

When you want to access secrets:

• Ensure the Azure service connection has at least Get and List permissions on the vault. You can set these permissions in the Azure
  portal:

  • Open the Settings blade for the vault, choose Access policies, then Add new.

  • In the Add access policy blade, choose Select principal and select the service principal for your client account.

  • In the Add access policy blade, choose Secret permissions and ensure that Get and List are checked (ticked).

  • Choose OK to save the changes.

Reference

Fresnel

Thank you Jayendran, will look further into this option – Yean

@MShareef let me know if you need any help. You are welcome to accept my answer – Fresnel

Answer 2

You can use

1. Secure Azure DevOps Variables or Variable Groups
2. Azure Key Vault
3. If you use a Service Principal, then you need a password / certificate as well to authenticate. Maybe you can also try to work with MSI (Managed Service Identity). In that case, the AAD will take care of the secret storage.

Harvest

Answer 3

If you don't want to store credentials on Azure Devops itself, best way is to store credentials in a credential store (Azure Key Vault) and access it through a service connection. I assume that you are using YAML based pipelines. If so use the following steps to integrate your pipeline with the key vault,

Prerequisites,

• Azure key vault is set up and keys are securely stored

Steps,

1. In edit mode of the pipeline click on the kebab menu (three dots on upper right corner) and select Triggers
2. On the opened menu click on the Variables tab and then Variable Groups
3. Open Manage variable groups in a new tab
4. Click on + Variable group button to add a new variable
5. Give a name and a description. Switch on the Link secrets from an Azure key vault as variables toggle.
6. Add a new service connection and once authenticated select the key vault name
7. Now add variables in to the variable group
8. Once done save the variable group and go back to the previous tab in step 2 and link the new variable group.
9. Once done save the pipeline

Important: You need to grant secret read permission to the service connection's service principal from your key vault. Reference: Link secrets from an Azure key vault

Entomb

Answer 4

Perhaps use the Azure Devops Libary > Variable Groups to securely store you keys.

Alternatively you may be able to use the Project Settings> Service connection. Perhaps using credentials connection or a generic on.

Beagle

Answer 5

Service principals is the industry standard for this case. You should create a specific service principal for Azure DevOps and limit its scope to only what's necessary.

Slime

Answer 6

1. You have to create a private key for Devops pipeline with limited services at your AWS machine
2. store the key in the Secure library of Devops Pipeline
3. from your AWS firewall disable the SSH connection from unknows IP addresses, and white-list Devops agents IP address, to get the list of the ips check this link https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=vsts&tabs=yaml#agent-ip-ranges

Abisia

Answer 7

you can write variables into your powershell script file and can use powershell task into your pipeline. Now give powershell file path into this task and just give variables names. It will work like a charm.

Aludel

Answer 8

For Service principle connection, you need to have service principle id and service principle key service principle id is same as application id service principle key is there in certificates and secrets

Dharma

Answer 9

You can use Azure Key Vault for storing all your keys and secrets. Give permission to your Azure pipeline to fetch keys from Key Vault.

Following link will guide you from scratch to develop a pipeline and fetch keys:

https://azuredevopslabs.com/labs/vstsextend/azurekeyvault/

Ludovico

Answer 10

The only method to truly not store AWS credentials in Azure/Azure DevOps would be to make a hosted build pool inside your AWS account. These machines will have the azure DevOps agent installed and registered to your Organization and to a specific agent pool. Then add the needed permissions to the Iam instance profile attached to these build servers. When running your terraform commands using this agent pool, terraform will have access to the credentials on the instance. The same concept works for a container based build pool in AWS ECS.

Disproportionate

Answer 11

You can use Managed identity in your pipeline to authenticate with the Azure Key Vault. You can read more on Managed Identity here and Azure Key Vault here

Whitsun

Where? You didn't link anything. – Shanonshanta

Where? Need more details on this. – Beluga