Using KeyVault secrets to override appsettings in Azure App Service and locally

Question

Attempting to retrieve secrets from KeyVault in a C# App Service.

Local machine:

• Visual Studio > Tools > Options > Azure Service Authentication - authenticated Azure account

• Likely use az login in the shell that you dotnet run if on vs code etc. Not Checked.

Azure

• App service blade:
  • Set App Service identity to System Assigned
• Keyvault blade:
  • Created KeyVault
  • Created Secret: Name = "Foo"
  • Given myself manage secrets access policy
  • Given App Service identity Get and List secret access policy

appsettings.json

...
"KeyVaultName" : "abc123",
"Secrets": {
    "One" : "@Microsoft.KeyVault(Secreturi=[uri to secret copied from Azure blade])"
}
...

Program.cs

...
using Azure.Extensions.AspNetCore.Configuration.Secrets;
using Azure.Identity;
...
public static IHostBuilder CreateHostBuilder(string[] args)
    {
        return Host.CreateDefaultBuilder(args)
            .ConfigureAppConfiguration((context, config) =>
            {
                var builtConfig = config.Build();
                var secretClient = new SecretClient(
                    new Uri($"https://{builtConfig["KeyVaultName"]}.vault.azure.net/"),
                    new DefaultAzureCredential());
                config.AddAzureKeyVault(secretClient, new KeyVaultSecretManager());
            })
            .ConfigureWebHostDefaults(webBuilder =>
            {
                webBuilder.UseStartup<Startup>();
            });
    }

Result

I am just getting the @Microsoft ... value which I had expected to be mapped to the value from the keyvault.

Something seems off as I have to define the name of the keyvault twice, once in the SecretClient and once in the @Microsoft.KeyVault reference.

Triumvirate

You don't need the @ tag to "reference" the secrets. Once you add the keyvault in your program (like you did) the values should become available when calling them from code. – Cobol

Answer 1

It seems I was mixing two methods of getting secrets from the KeyVault.

Configuration Provider

What I added in Program.cs was a configuration provider that maps secrets into the configuration collection. Putting a breakpoint in Startup.cs and inspecting the value in the configuration collection validated this.

What I should have done is named the secret Secret--One which will map and override the local config value { "Secret: { "One" : "..." } }. Cannot use : or __ used in Environment Variable config mapping as those characters are not supported in secret names.

Feel I am still missing something here so please update in comments or another answer.

KeyVault Reference in App Settings

If, on the other hand, you want to override config values using Environment Variables set on the Azure Application Settings (App Service Configuration) blade, then you can use KeyVault References.

The issue with this is that you still need another method to ensure you don't keep secrets locally and risk committing them to source control.

For more information on this method see answer by Enrico.

References

• Microsoft Docs - KeyVault References
• Microsoft Docs - KeyVault Configuration
• Joseph Guandagno - Example using KeyVault Reference in Application Settings blade

Triumvirate

Key Vault references must be setup in App Service Application Settings, not in your configuration files. App Service then resolves them and offers the values to your app as environment variables. The Key Vault configuration provider on the other hand loads all secrets from Key Vault at startup and adds them to the in-memory configuration (with the special handling for configuration sections that you noted). – Footlight

The double dash is what got it to work for me with the Options Pattern. – Selfimprovement

Answer 2

You can use a keyvault reference without any nuget packages by referencing it in your appsettings on the app service.

Simple copy the "uri" of your secret from keyvault by clicking on the copy icon

In your appsettings under application settings add a new setting and use this syntax to reference a keyvault secret.

@Microsoft.KeyVault(SecretUri=https://myvault.vault.azure.net/secrets/mysecret/)

Step by step guide here

Cobol

Thanks for giving full instructions on how to use this second method @Cobol ... sure that will help someone. Note though that this does not work on the local developer machine. You will need to use a different method there such as Secret Manager. Further reading at learn.microsoft.com/en-us/aspnet/core/security/app-secrets – Triumvirate

You are 100% correct @Ruskin, if you want to use keyvault locally you do need to install the package. I would recommend using secrets however. (Right click -> manage secrets) – Cobol

Right click project in Visual Studio and choose Manage User Secrets in context menu I assume. – Triumvirate

@Triumvirate correct, or use dotnet user-secrets set "mykey" "secret value" – Cobol

you are linking to a guide that requires paid subsrcription? – Serration

@MaythamFahmi it's not behind a paywall. The essence is written in the stackoverflow post. – Cobol

Answer 3

What I like to do is let the normal Configure happen for a strongly types object first. So if I have a config with secrets in it, but it also has stuff that's fine to leave in a config like this:

"ComplianceSettings": {
"Url": "https://www.somecomplianceservice.com",
"Password": "Stored In Azure key vault",
"UserName": "Stored In Azure key vault"
}

So I let that read in normally with:

var complianceSettingsConfig = configuration.GetSection(Constants.ComplianceSettingsKey); // constant value is "ComplianceSettings"
services.Configure<ComplianceSettings>(complianceSettingsConfig);

ComplianceSettings is just a Poco with Url, Password, UserName properties

Then as long as I have my program.cs setup correctly like this:

.ConfigureAppConfiguration((context, builder) =>
{
           var configuration = builder.Build();

               var azureServiceTokenProvider = new AzureServiceTokenProvider();
               var keyVaultClient = new KeyVaultClient(
                   new KeyVaultClient.AuthenticationCallback(
                       azureServiceTokenProvider.KeyVaultTokenCallback));

               builder.AddAzureKeyVault(
                   configuration["KeyVault:KeyVaultUrl"],
                   keyVaultClient,
                   new DefaultKeyVaultSecretManager());
 })

The "KeyVault:KeyVaultUrl" is just another config entry in appsettings.json as section:key like this:

"KeyVault": {
"KeyVaultUrl": "https://<your-url-to-kv>.vault.azure.net/"
}

If you inject IOptions in your controller's constructor, you'll see that you have all your local values and NONE from the Azure KeyVault. I think this is your actual question.... If not, then maybe this helps someone else ;)

What's happening is that all this services.Configure() stuff happens before the Azure data is there. So we need to hook the PostConfigure to overwrite the values from the keyvault like this:

services.PostConfigure<ComplianceSettings>(options =>
{
    options.UserName = configuration.GetValue<string>(Constants.ComplianceUserNameKey);
    options.Password = configuration.GetValue<string>(Constants.CompliancePasswordKey);
 });

Constants.ComplianceUserNameKey and Constants.CompliancePasswordKey hold the name of the actual key in the keyvault. So basically, when PostConfigure fires, it looks in configuration.GetValue for a key you specify. This time the keyvault secrets have been loaded into configuration, so you just look them up by the name they have in the keyvault and overwrite the properties in your strongly typed object with the new values.

I generally use an extension method like:

public static class ServicesExtensions
{
    public static IServiceCollection AddConfiguration(this IServiceCollection services, IConfiguration configuration)
    {
       // All configuration gets loaded here
       // same as above, just gets PostConfigure if it needs
       // to load secrets, if not its just a GetSection() and Configure()
    }
 }

Then just add to ConfigureServices

services.AddConfiguration(Configuration);

Hope that helps someone!

Gnash

Thanks Jeff ... you don't need the PostConfigure step ... just name your secrets ComplianceSettings--UserName and ComplianceSettings--Password and the configuration cascade will map them correctly. – Triumvirate

I did not know that! Thanks Ruskin! Can't wait to try it!! – Gnash