You can restrict the access for a service account to a specific bucket using Cloud IAM.
This is the gsutil command you can use:
gsutil iam ch serviceAccount:[email protected]:objectAdmin gs://my-project/my-bucket
for custom roles use
gsutil iam ch serviceAccount:[email protected]:projects/{your_project_id}/roles/{your_custom_role_id} gs://my-bucket
To remove a service account from all roles on a bucket:
gsutil iam ch -d serviceAccount:[email protected] gs://my-project/my-bucket
Or you can control access to buckets and objects using ACLs.
For example grant the service account WRITE (R: READ,W: WRITE,O: OWNER) access to the bucket:
gsutil acl ch -u [email protected]:W gs://my-project/my-bucket
To remove access of service account from the bucket:
gsutil acl ch -d [email protected] gs://my-project/my-bucket
I would suggest to remove the access of the service account from the buckets. Then grant access to a specific bucket.