Is it possible track the number of docker pulls in Google Artifact Registry?

Question

I'd like to measure the number of times a Docker image has been downloaded from a Google Artifact registry repository in my GCP project.

Is this possible?

Godevil

Answer 1

Interesting question.

I think this would be useful too.

I think there aren't any Monitoring metrics (no artifactregistry resource type is listed nor metrics are listed)

However, you can use Artifact Registry audit logs and you'll need to explicitly enable Data Access logs see e.g. Docker-GetManifest.

NOTE I'm unsure whether this can be achieved from gcloud.

Monitoring Developer tools, I learned that Audit Logs are configured in Project Policies using AuditConfig's. I still don't know whether this functionality is available through gcloud (anyone?) but evidently, you can effect these changes directly using API calls e.g. projects.setIamPolicy:

gcloud projects get-iam-policy ${PROJECT}
auditConfigs:
- auditLogConfigs:
  - logType: DATA_READ
  - logType: DATA_WRITE
  service: artifactregistry.googleapis.com
bindings:
- members:
  - user:me
  role: roles/owner
etag: BwXanQS_YWg=

Then, pull something from the repo and query the logs:

PROJECT=[[YOUR-PROJECT]]
REGION=[[YOUR-REGION]]
REPO=[[YOUR-REPO]]

FILTER="
logName=\"projects/${PROJECT}/logs/cloudaudit.googleapis.com%2Fdata_access\"
protoPayload.methodName=\"Docker-GetManifest\"
"

gcloud logging read "${FILTER}" \
--project=${PROJECT} \
--format="value(timestamp,protoPayload.methodName)"

Yields:

2022-03-20T01:57:16.537400441Z  Docker-GetManifest

You ought to be able to create a logs-based metrics for these too.

Jeffreyjeffreys

This was the first thing I tried, but no luck so far. For some reason there are no Docker-GetManifests in the audit logs. I see logs for ListRepositories, possibly only for logged in users in my organization and not other users of the docker registry. (Didn't verify this - just eyeballed the first ten entries or so.) I don't think it's an IAM issue as I seem to have appropriate permissions. Any ideas for troubleshooting this? – Godevil

It works for me. You must enable Audit Logs for Artifact Registry for Data Read|Write. I'll add an example log query to my answer. – Jeffreyjeffreys

No luck, still doesn't work. Is it possible this doesn't work for public images? Is there support I can reach out to? – Godevil

It would be straightforward for you to test your hypothesis; create a non-public image and see whether the audit logs are reported for it alone. If you Google "Google Cloud Support", the first result is Google Cloud Support. You can also file an issue using Google's public issue tracker. – Jeffreyjeffreys

Yeah, true. Will do thanks. – Godevil

Note if someone has viewer role they can see 99% of logs, audit logs is one of those things that can need additional rights. – Auctorial

Answer 2

We do not yet have platform logs for Artifact Registry unfortunately, so using the CALs is the only way to do this today. You can also turn the CALs into log-based metrics and get graphs and metrics that way too.

The recommendation to filter by 'Docker-GetManifest' is also correct - it's the only request type for which a Docker Pull always has exactly one. There will be a lot of other requests that are related but don't match 1:1. The logs will have all requests (Docker-Token, 0 or more layer pulls), including API requests like ListRepositories which is called by the UI in every AR region when you load the page.

Unfortunately, the theory about public requests not appearing is correct. CALs are about logging authentication events, and when a request has no authentication whatsover, CALs are not generated.

Crabber