Cloud Build fails to deploy to Google App Engine - You do not have permission to act as @appspot.gserviceaccount.com

Question

This morning I made a PR which triggered a Cloud Build for my staging enviroment but failed to deploy the results to GAE.

The error was as follows:

ERROR: (gcloud.app.deploy) PERMISSION_DENIED: You do not have permission to act as '[redacted]@appspot.gserviceaccount.com' Step #4: - '@type': type.googleapis.com/google.rpc.ResourceInfo Step #4: description: You do not have permission to act as this service account. Step #4: resourceName: [redacted]@appspot.gserviceaccount.com Step #4: resourceType: serviceAccount

When I look at https://console.cloud.google.com/cloud-build/settings/service-account Cloud build has the follow service account permissions ENABLED:

• App Engine Admin
• Cloud KMS

Checking https://console.cloud.google.com/iam-admin/iam I can see that the cloudbuild service account has the following roles:

• App Engine Admin
• App Engine Deployer
• Cloud Build Service Account
• Cloud KMS CryptoKey Decrypter

Zrike

Hi @LawsonTaylor considering the error message you are seeing, it might be related to the fact that the default Cloud Build service account does not allow access to deploy App Engine. Could you please follow the steps here to give deployer permission to your Cloud Build service account? – Evelunn

@Evelunn For my projects, this has been working fine for quite some time, but stopped working this morning. This doc may need to be updated: cloud.google.com/cloud-build/docs/deploying-builds/… - I only had the "App Engine Admin" permission as indicated by the doc. I added the "App Engine Deployer" IAM Permission as your link suggested, and it still doesn't work. – Cavort

Just to add more details, this is definitely a recent change/regression in GCP. My build account previously had the App Engine Deployer role, but started failing with a recent build. I had to use @Nebulastic 's answer to fix. Would be nice if the App Engine team could comment with a bug number - seems very strange that having "App Engine Deployer" role alone is no longer enough to actually deploy App Engine. – Peggi

Answer 1

According to the provided error, it seems like you need to add some delegation to your service account. This means that the service account can act on behalf of another service account. Do not add this permission on the project level, since it poses a security risk! Below you can find an example of how to add roles/iam.serviceAccountUser on another service account.

PROJECT_ID=xxxxxx

PROJECT_NUMBER=$(gcloud projects list \
  --format="value(projectNumber)" \
  --filter="projectId=${PROJECT_ID}")

gcloud iam service-accounts add-iam-policy-binding \
    ${PROJECT_ID}@appspot.gserviceaccount.com \
    --member=serviceAccount:${PROJECT_NUMBER}@cloudbuild.gserviceaccount.com \
    --role=roles/iam.serviceAccountUser \
    --project=${PROJECT_ID}

To summarize, the service account must have the iam.serviceAccounts.actAs permission, which is included in the roles/iam.serviceAccountUser role. Updated Google documentation can be found here.

Debtor

Since this morning (2020-10-09) some of our deployment started to fails and this solved it. In our case we are using: gcloud --quiet --project "$GOOGLE_PROJECT_NAME" app deploy app.yaml and the deployment have previously been working for months. – Concinnity

If you get this error, ERROR: (gcloud.iam.service-accounts.add-iam-policy-binding) INVALID_ARGUMENT: The member <project-number>@cloudbuild.gserviceaccount.com is of an unknown type. Please set a valid type prefix for the member. insert serviceAccount: before <project-number> – Voltage

Updated the snippet so you don't have to look up the project number anymore. – Debtor

@Nebulastic The commands I'm using are here -- gitlab.com/apgoucher/catagolue/-/blob/master/… -- and the failed job output is here -- gitlab.com/apgoucher/catagolue/-/jobs/783798218 Shih-En Chou's solution of adding Service Account User to the gitlab service account solved the problem. – Jetton

Yes but your permissions are on the project level and should be on the service account level. Saying "it works" is not always the best solution. – Debtor

Where do we add this part? I am quite a beginner, so I am not sure where to add this. – Batchelor

You can execute this on the command line with the gcloud sdk. – Debtor

I have filled an issue on Google issue tracker (see issuetracker.google.com/issues/170538212) for this issue and in the response Google confirmed the serviceAccountUser role is now required. They also updated the documentation at cloud.google.com/appengine/docs/flexible/nodejs/… – Neoteny

Yep, this is because of cloudbuild poses a possible security risk, so I bet they are closing off some known vulnerabilities. – Debtor

@Nebulastic Thanks for this answer. Please, where are we supposed to run this command, on the local machine or CD pipeline? I ran it on my local machine, and rerun the workflow but it's still spitting out the error. – Wanderjahr

Please be more specific about your error, so others can help. You could run this command locally or in your pipeline. The same logic can also be applied in Deployment Manager or Terraform if you use those. – Debtor

I got this error from Github Action: ERROR: (gcloud.app.deploy) PERMISSION_DENIED: You do not have permission to act as ‘[email protected]’ - ‘@type’: type.googleapis.com/google.rpc.ResourceInfo description: You do not have permission to act as this service account. resourceName: [email protected] resourceType: serviceAccount So I came here, got your answer and ran the command on the terminal on my machine. Afterwards, I retried the workflow but the error above still occurred. I am deploying to App Engine. – Wanderjahr

You have to set the permissions for the service account you are using in github actions, to "act as" the appspot service account. – Debtor

Thanks for the reply. Please, how do I do that? – Wanderjahr

Never mind, I found the answer here: https://mcmap.net/q/323582/-how-do-you-enable-quot-iam-serviceaccounts-actas-quot-permissions-on-a-sevice-account. Thanks, man! – Wanderjahr

@OldrichDlouhy thanks for filing the issue. It is pretty annoying that they would make these changes without more notice. – Anthocyanin

Answer 2

I had the same issue. For me I had to add the Service Account User role to my circle ci user in IAM. Maybe you can do the same for cloudbuild.

Agrostology

This worked for me too, it looks like something has changed but the docs are up to date with the roles required for automation. cloud.google.com/appengine/docs/standard/python/roles – Rachealrachel

Just giving the Service Account the "Service Account User" role is too broad, and basically gives admin right on the project to your deployer/cloudbuild service account. One should use gcloud iam service-accounts add-iam-policy-binding to bind the role only to the '[redacted]@appspot.gserviceaccount.com' SA. See answer from Nebulastic. – Concinnity

Totally agree with @Mayeu, this way your cloudbuild service account can perform almost any action within the project, not a best practice. The scope should be only the service account for which permissions need to be delegated. – Debtor

I user CircleCI and approve of this solution – Qumran

Although this would work, you are creating a serious security issue here. Never set the Service Account User role in the project IAM, since it allows a user account to act as any other user account. – Debtor

Nebulastic feels strongly enough to post the same comment about over-permission-ing in 3 different places in this thread. While I agree with the point - so far no one has posted a viable alternative solution ... this means people will over-permission, until an alternative is proposed. – Subsoil

I am actually very confident that you should not put this as a project role, but as a service account binding! This is exactly what I described in my post. – Debtor

Answer 3

First we go to the permission manager and select the project that we want to add permissions.; https://console.cloud.google.com/iam-admin/

Parsley

Answer 4

I grant Service Account User permission to my CI/CD service account. That works.

Screenshot of IAM

Screenshot of my Gitlab CI/CD configuration

Gerund

This is basically over-permissioning, since the service account can now act on behalf of all the other service accounts within the project. See my answer for the solution. – Debtor

This it too much over-permissioning. Cant do in prod – Panoply

Answer 5

To resolve this issue, you can add Service Account User IAM permission to your CI/CD pipeline service account.

Eg. If you're using Cloud Build, then add Service Account User role to your {project-number}@cloudbuild.gserviceaccount.com service account

Hardee

This role is way too broad. You should only specify it on the service account level. Otherwise, anyone using cloudbuild can act as another service account, posing serious security issues. – Debtor

Answer 6

Terraform way:

Granting service account user role to cloud sv account only on the GAE service account

data "google_app_engine_default_service_account" "app_engine_sv_account" {
  project = var.project_id
}

resource "google_service_account_iam_member" "cb-deploy-iam" {
  service_account_id = data.google_app_engine_default_service_account.app_engine_sv_account.name
  role               = "roles/iam.serviceAccountUser"
  member             = "serviceAccount:${var.cb_sv_account_email}"
}

Panoply

Answer 7

It looks as though this question is answered with the .ActAs permission being added to the Gitlab or CircleCI account.

I haven't had occasion to test yet - if anyone else has and can post details - please do so;

This is the proposed answer from what I can gather: How do you enable "iam.serviceAccounts.actAs" permissions on a sevice account?

Nebulastic has a very nice answer above but the {PROJECT_ID} would need to be swapped with the Gitlab or CircleCI account name, not the project named account.

Subsoil

Arguably it is NOT clear - since I discovered this post via Google Fu which I am SURE others have as well... hence my sharing of helpful information highly related to the post in the spirit of supporting the community, @Nebulastic. – Subsoil