Cloud composer: "PERMISSION_DENIED: The caller does not have permission"

Question

I implemented a few tasks with BashOperator. Ones with "gsutil rm" and "gsutil cp" worked fine. But one with "gcloud alpha firestore export" generates this error:

{bash_operator.py:101} INFO - ERROR: (gcloud.alpha.firestore.export) PERMISSION_DENIED: The caller does not have permission

This command itself works fine in gcloud shell. I tried to give some Firestore related permissions to the service account used by the Composer but it still doesn't work. Any idea

Codeclination

Answer 1

Try creating a new service account with the Firestore-related permissions needed and using that on a freshly created environment. https://cloud.google.com/composer/docs/how-to/access-control

Other debugging ideas: * Try ssh-ing into the Kubernetes workers on your Composer environment and running the command. *Is the Firestore API enabled on your project?

Elflock

Answer 2

It might be that you don't have permissions for a particular project.

The error I was getting was: PERMISSION_DENIED: Caller does not have required permission to use project project:random-id-11111.

The way I resolved it was by running gcloud config set project 'the-right-project-id' and then the actual gcloud command.

Practicable

Answer 3

I think you need Cloud Datastore Import Export access. Following are the steps as per current Current Google Cloud platform layout.

https://console.cloud.google.com > Left drawer > IAM & admin > Against user - Edit Icon > Add another role > Data Store > Cloud Datastore Import & Export > Save

Maddalena

Answer 4

1. Open https://console.cloud.google.com/iam-admin/iam
2. Find the service account you're using for the backups
3. Add the Owner role to the service account

It's not really intuitive or logic because there are not permissions or roles for Firestore.

Unfortunately it took me way to long to figure it out. I hope it helps others!

Tertia

firebase.google.com/docs/firestore/manage-data/export-import it has more defined permissions for firestore export – Severus

Do not give any service account Owner role ever. You need the roles/cloudtrace.agent role. cloud.google.com/trace/docs/iam#roles – Porshaport

I think this has changed since 2018. I believe the owner role is not needed anymore. – Tertia

Answer 5

Similar to Roy's answer, the issue for me was that gcloud was set to a different project.

check which project it is set to

gcloud config list

list which projects you have access to

gcloud projects list

set the correct project

gcloud config set project 'foo-project'

Aristippus

worked for me after setting up the right project where i want to update the composer environment. – Cwmbran

Answer 6

Grant firebase admin role to the default service account that your service is using.

Adding Owner role to the service account seems too much privilege for just taking backup.

Savannahsavant

Answer 7

In IAM & Admin make sure your @appspot.gserviceaccount.com must have access for 3 things:

• Cloud Functions Admin
• Cloud Datastore Import Export Admin
• Storage Admin

Anyaanyah

Answer 8

you need to set your project first where you are owner gcloud config set project project-id You can find your project id by clicking on gcloud console it will be there in popup in project-name-somerandomnumbers

Grateful

Answer 9

I got caught out on this today. The issue was that I had set up my service account correctly in the IAM settings, but hadn't realised an invitation had been sent to that email address which I needed to accept. Worked immediately once I accepted the invite.

Bolzano

Answer 10

If you haven't created any firebase project before, go to firebase, try to create any project and... accept terms. It seems that it was problem in my case.

{ "error": { "code": 403, "message": "The caller does not have permission", "status": "PERMISSION_DENIED" } }

Duprey