Permission 'cloudkms.cryptoKeyVersions.useToDecrypt' denied for resource ...key

Question

I am building an http endpoint with Google Cloud Functions. I have an encrypted secret stored as a file that is loaded and decrypted in the function as a way to prevent my secret from being stored in the code. Usually I dynamically load something from Google Cloud Storage but it seems like KMS is more meant for this purpose.

The code that uses KMS looks like this:

getCredentials: async function () {
    const kms = require('@google-cloud/kms');
    const client = new kms.KeyManagementServiceClient();
    const fs = require('fs');
    let ciphertext = (fs.readFileSync('secret.enc')).toString('base64')
    const name = client.cryptoKeyPath(
      '[project]',
      'global',
      '[keyring]',
      '[key]'
    );

Everything runs fine locally but I can't seem to get the function to work when called with the http trigger. Checking the logs I see this:

textPayload:  "Error: Permission 'cloudkms.cryptoKeyVersions.useToDecrypt' denied for resource 'projects/[projectname]/locations/global/keyRings/[keyring]/cryptoKeys/[key]'.
    at Http2CallStream.call.on (/srv/functions/node_modules/@grpc/grpc-js/build/src/client.js:96:45)
    at Http2CallStream.emit (events.js:194:15)
    at Http2CallStream.EventEmitter.emit (domain.js:459:23)
    at process.nextTick (/srv/functions/node_modules/@grpc/grpc-js/build/src/call-stream.js:71:22)
    at process._tickCallback (internal/process/next_tick.js:61:11)" 

I have tried all sorts of IAM permissions (including owner) with on avail so it seems like I must have a deeper misunderstanding.

This is potentially related to another issue I have where I cannot get Google Cloud Build to deploy the function. It errors without help:

starting build "b2321cdb-bd4c-4828-8d38-80a86f4fe808"

FETCHSOURCE
Initialized empty Git repository in /workspace/.git/
From https://source.developers.google.com/p/[projectname]/r/[repo]
* branch 314691d6e63199caf867c74bcd0090bc70386a0e -> FETCH_HEAD
HEAD is now at 314691d Merge pull request #2 from [repo]/tristans/update-deploy-cloudbuild
BUILD
Already have image (with digest): gcr.io/cloud-builders/gcloud
Deploying function (may take a while - up to 2 minutes)...
...............failed.
ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Build failed: Build error details not available
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/gcloud" failed: exit status 1

It doesn't seem like you would need any KMS permissions to deploy functions deploy name --trigger-http --runtime=nodejs10 --entry-point=fname --project=project and like I said it works fine when I run the gcloud deploy locally so I'm not sure why this would fail. We have several cloud functions with similar deployment processes set up so it seems like there is something non-obvious or broken about how KMS works that avails me but perhaps it is a red herring.

If there is a better way to use KMS for this purpose, I am all ears!

Barrister

Can you share your code? What commands did you run? What service account are you attaching to Cloud Build / Run? – Mun

@Mun I added in a snippet with the KMS stuff, like I said the code runs fine when called locally. I have been adjusting the IAM permissions for my default Google Cloud Functions Service Agent and Cloud Build Service Account . Do you suggest creating new service accounts? I'm not sure how to run cloud functions in the wild on arbitrary service accounts. – Barrister

I'm looking for a gcloud functions deploy call or something. Have you looked at github.com/sethvargo/secrets-in-serverless and github.com/GoogleCloudPlatform/berglas by chance? – Mun

It's difficult to say without seeing more, but the GCF service account isn't what needs to be adjusted, it's the Google Compute Engine default service account. GCF runs as the GCE service account by default. You should be able to find success in one of the links I posted above. Just in case, here's an even more in-depth dive: cloud.google.com/blog/products/application-development/… – Mun

Thank you for your help! gcloud functions deploy [name] --trigger-http --runtime=nodejs10 --entry-point=[name] --project=[project] is my deploy command on both cloud build and local. I will review the above resources and let you know if I can get it to work. It seems unintuitive that GCF would run as GCE service accounts by default. What would be the purpose of having the GCF accounts then? – Barrister

@Mun creating a new service account with decrypt IAM permissions and adding it to de deploy script did the trick. Thank you so much for your help! – Barrister

I ran into the same issue on a CI job, and doubled checked everything with no success. In the end, creating a new service-account with same settings made the setup eventually work. But when looking closer, I realized that I had changed something along the way, and after verification it was the real culprit for me : setting GOOGLE_APPLICATION_CREDENTIALS to the path for my service-account credentials json made all the difference. I hope this comment may help someone in a similar situation. – Bubal

Answer 1

Per Google Functions docs, function's runtime service account is [email protected]
You need to grant KMS decrypt permissions to this account.
This can be done under "IAM & Admin / IAM" section of Google cloud web console

Separatist

Thank you so much @Slava Medvediev Issue has been resolved because of your comment – Ripon

Answer 2

Just restart it's worked for me

or

check your IAM access it should be insufficient permission

Meteorite