How to use a Google Secret in a deployed Cloud Run Service (managed)?

Question

I have a running cloud run service user-service. For test purposes I passed client secrets via environment variables as plain text. Now since everything is working fine I'd like to use a secret instead.

In the "Variables" tab of the "Edit Revision" option I can declare environment variables but I have no idea how to pass in a secret? Do I just need to pass the secret name like ${my-secret-id} in the value field of the variable? There is not documentation on how to use secrets in this tab only a hint at the top:

Store and consume secrets using Secret Manager

Which is not very helpful in this case.

Neuron

Answer 1

UPDATE 2021: There is now a Cloud Run preview for loading secrets to an environment variable or a volume. https://cloud.google.com/run/docs/configuring/secrets

The question is now answered however I have been experiencing a similar problem using Cloud Run with Java & Quarkus and a native image created using GraalVM.

While Cloud Run is a really interesting technology at the time of writing it lacks the ability to load secrets through the Cloud Run configuration. This has certainly added complexity in my app when doing local development.

Additionally Google's documentation is really quite poor. The quick-start lacks a clear Java example for getting a secret[1] without it being set in the same method - I'd expect this to have been the most common use case!

The javadoc itself seems to be largely autogenerated with protobuf language everywhere. There are various similarly named methods like getSecret, getSecretVersion and accessSecretVersion

I'd really like to see some improvment from Google around this. I don't think it is asking too much for dedicated teams to make libraries for common languages with proper documentation.

Here is a snippet that I'm using to load this information. It requires the GCP Secret library and also the GCP Cloud Core library for loading the project ID.

public String getSecret(final String secretName) {
    LOGGER.info("Going to load secret {}", secretName);

    // SecretManagerServiceClient should be closed after request
    try (SecretManagerServiceClient client = buildClient()) {
        // Latest is an alias to the latest version of a secret
        final SecretVersionName name = SecretVersionName.of(getProjectId(), secretName, "latest");
        return client.accessSecretVersion(name).getPayload().getData().toStringUtf8();
    }
}

private String getProjectId() {

    if (projectId == null) {
        projectId = ServiceOptions.getDefaultProjectId();
    }

    return projectId;
}

private SecretManagerServiceClient buildClient() {
    try {
        return SecretManagerServiceClient.create();
    } catch(final IOException e) {
        throw new RuntimeException(e);
    }
}

[1] - https://cloud.google.com/secret-manager/docs/reference/libraries

Brazee

+1 for the sake of google reading this. However be prepared to be destroyed by the SO Inquisition of Moderators – Neuron

if you don't want to change your code, you can either configure your app to use these secrets from the dockerfile itself as in this amazing article or use cloud build as in this documentation – Chantay

Answer 2

You can now read secrets from Secret Manager as environment variables in Cloud Run. This means you can audit your secrets, set permissions per secret, version secrets, etc, and your code doesn't have to change.

You can point to the secrets through the Cloud Console GUI (console.cloud.google.com) or make the configuration when you deploy your Cloud Run service from the command-line:

gcloud beta run deploy SERVICE --image IMAGE_URL --update-secrets=ENV_VAR_NAME=SECRET_NAME:VERSION

Six-minute video overview: https://youtu.be/JIE89dneaGo

Detailed docs: https://cloud.google.com/run/docs/configuring/secrets

Crites

When I use --update-secrets I get this error - gcloud crashed - Invalid secret path even though the secret is valid within the project where cloud run is being deployed. – Fallfish

Answer 3

Google have documentation for the Secret manager client libraries that you can use in your api.

This should help you do what you want https://cloud.google.com/secret-manager/docs/reference/libraries

Since you haven't specified a language I have a nodejs example of how to access the latest version of your secret using your project id and secret name. The reason I add this is because the documentation is not clear on the string you need to provide as the name.

 const [version] = await this.secretClient.accessSecretVersion({
        name: `projects/${process.env.project_id}/secrets/${secretName}/versions/latest`,
      });
 return version.payload.data.toString()

Be sure to allow secret manager access in your IAM settings for the service account that your api uses within GCP.

Plebs

Ah. Well I thought Google will pass the secret value as a environment variable. Therefore I thought I can somewhat connect it in the Cloud Console rather than fetching it via code. – Neuron

Yeah not to my knowledge. You have to provide the secret name yourself in your env settings and pass that to your secret manager (SM) client library or else GCP won't know what secret you want. IF you only have one secret though, you could use the SM to list all your secrets and then pick the first one. That would mean you wouldn't have to worry about env vars. However, this would bring maintenance issues if you added another secret. – Plebs

Answer 4

I kinda found a way to use secrets as environment variables.

The following doc (https://cloud.google.com/sdk/gcloud/reference/run/deploy) states:

Specify secrets to mount or provide as environment variables. Keys starting with a forward slash '/' are mount paths. All other keys correspond to environment variables. The values associated with each of these should be in the form SECRET_NAME:KEY_IN_SECRET; you may omit the key within the secret to specify a mount of all keys within the secret. For example: '--update-secrets=/my/path=mysecret,ENV=othersecret:key.json' will create a volume with secret 'mysecret' and mount that volume at '/my/path'. Because no secret key was specified, all keys in 'mysecret' will be included. An environment variable named ENV will also be created whose value is the value of 'key.json' in 'othersecret'. At most one of these may be specified

Neuron

Unfortunately, that only seems to work for Anthos: Running gcloud run deploy --help shows that option under the category: Only applicable if connecting to Cloud Run for Anthos deployed on Google Cloud or Cloud Run for Anthos deployed on VMware. Specify --platform=gke or --platform=kubernetes to use: – Mccloskey

It really is a disgrace that Google is making it so painful to access those secrets. It wouldn't kill them to provide a gcloud run deploy option that simply makes all secrets in my project available as environment variables of the same name. – Mccloskey

As of yesterday, you can now read secrets as environment variables in managed (non-Anthos) Cloud Run as well. Find the details at cloud.google.com/run/docs/configuring/secrets – Crites

Answer 5

Here is a snippet of Java code to get all secrets of your Cloud Run project. It requires the com.google.cloud/google-cloud-secretmanager artifact.

Map<String, String> secrets = new HashMap<>();          
String projectId;
String url = "http://metadata.google.internal/computeMetadata/v1/project/project-id";
HttpURLConnection conn = (HttpURLConnection)(new URL(url).openConnection());
conn.setRequestProperty("Metadata-Flavor", "Google");
try {
   InputStream in = conn.getInputStream();
   projectId = new String(in.readAllBytes(), StandardCharsets.UTF_8);
} finally {
   conn.disconnect();
}               
Set<String> names = new HashSet<>();
try (SecretManagerServiceClient client = SecretManagerServiceClient.create()) {
   ProjectName projectName = ProjectName.of(projectId);
   ListSecretsPagedResponse pagedResponse = client.listSecrets(projectName);
    
   pagedResponse
      .iterateAll()
      .forEach(secret -> { names.add(secret.getName()); });
   for (String secretName : names) {
      String name = secretName.substring(secretName.lastIndexOf("/") + 1);
      SecretVersionName nameParam = SecretVersionName.of(projectId, name, "latest");
      String secretValue = client.accessSecretVersion(nameParam).getPayload().getData().toStringUtf8();
      secrets.put(secretName, secretValue);
   }
}

Mccloskey

In combination with Spring's @DynamicPropertySource this is not too bad to use – Neuron

Answer 6

Cloud Run support for referencing Secret Manager Secrets is now at general availability (GA).

https://cloud.google.com/run/docs/release-notes#November_09_2021

Rhumb

While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - From Review – Ballad