Verify HTTP request from Google Cloud Scheduler

Question

What's the process for verifying the HTTP request from Google Cloud scheduler? The docs (https://cloud.google.com/scheduler/docs/creating) mention you can create a job with a target of any publicly available HTTP endpoint but do not mention how the server verifies the cron/scheduler request.

Mithridate

Answer 1

[Update May 28, 2019]

Google Cloud Scheduler now has two command line options:

--oidc-service-account-email=<service_account_email>
--oidc-token-audience=<service_endpoint_being_called>

These options add an additional header to the request that Cloud Scheduler makes:

 Authorization: Bearer ID_TOKEN

You can process the ID_TOKEN inside your endpoint code to verify who is calling your endpoint.

For example, you can make an HTTP request to decode the ID Token:

https://oauth2.googleapis.com/tokeninfo?id_token=ID_TOKEN

This will return JSON like this:

{
  "aud": "https://cloudtask-abcdefabcdef-uc.a.run.app",
  "azp": "0123456789077420983142",
  "email": "[email protected]",
  "email_verified": "true",
  "exp": "1559029789",
  "iat": "1559026189",
  "iss": "https://accounts.google.com",
  "sub": "012345678901234567892",
  "alg": "RS256",
  "kid": "0123456789012345678901234567890123456789c3",
  "typ": "JWT"
}

Then you can check that the service account email matches the one that you authorized Cloud Scheduler to use and that the token has not expired.

[End Update]

You will need to verify the request yourself.

Google Cloud Scheduler includes several Google specific headers such as User-Agent: Google-Cloud-Scheduler. Refer to the documentation link below.

However, anyone can forge HTTP headers. You need to create a custom something that you include as an HTTP Header or in the HTTP body that you know how to verify. Using a signed JWT would be secure and easy to create and verify.

When you create a Google Cloud Scheduler Job you have some control over the headers and body fields. You can embed your custom something in either one.

Scheduler Jobs

[Update]

Here is an example (Windows command line) using gcloud so that you can set HTTP headers and the body. This example calls Cloud Functions on each trigger showing how to include an APIKEY. The Google Console does not have this level of support yet.

gcloud beta scheduler ^
--project production ^
jobs create http myfunction ^
--time-zone "America/Los_Angeles" ^
--schedule="0 0 * * 0" ^
--uri="https://us-central1-production.cloudfunctions.net/myfunction" ^
--description="Job Description" ^
--headers="{ \"Authorization\": \"APIKEY=AUTHKEY\", \"Content-Type\": \"application/json\" }" ^
--http-method="POST" ^
--message-body="{\"to\":\"/topics/allDevices\",\"priority\":\"low\",\"data\":{\"success\":\"ok\"}}"

Imaimage

thanks! Just a note on this, currently the Google Cloud Platform console UI only allows you to set a custom POST/PUT body. The ability to set custom HTTP headers is not available in the UI. – Mithridate

@Mithridate - I just updated my answer to show a gcloud example setting HTTP headers and body. I hope that the Google Console has this support when released from beta. – Imaimage

This statement However, anyone can forge HTTP headers is incorrect. Don't count on this, but I've read somewhere, that Google strips all headers with forbidden prefixes, for example (I might be mistaken) X-Google, and so on – Cosmic

@Prometheus - I forge headers with curl and Python scripts everyday including requests to Cloud Functions. Part of our vulnerability testing for public facing software and APIs. If you have a reference for you comment, please add it. – Imaimage

@johnhanley Sure, groups.google.com/forum/#!topic/google-appengine/FAxqswxW4dk This is slightly different problem, but the gist is the same - verifying the origin of requests – Cosmic

@Prometheus - not one of the headers in my answer is covered by your reference link. – Imaimage

@johnhanley Please, take a look at my answer – Cosmic

Isn't this ID_TOKEN easily forged too? Wouldn't you need the signing key's pair to verify it? – Ilysa

@GerardVuyk - To forge a Google Identity Token would require the Private Key from Google's signing certificate. Google publishes the Public Keys so you can verify Google signatures. – Imaimage

When configuring an AppEngine HTTP target there is no option to specify security tokens, it no longer sets special x-google or x-appengine headers, and it now sets x-cloudscheduler and x-cloudscheduler-jobname headers. Those two headers are not protected by GCP meaning anyone can set them. No documentation on how to assume safety so this has really made things hard. – Gerladina

Answer 2

Short answer

If you host your app in Google Cloud, just check if header X-Appengine-Queuename equals __scheduler. However, this is undocumented behaviour, for more information read below.

Furthermore, if possible use Pub/Sub instead of HTTP requests, as Pub/Sub is internally sent (therefore of implicitly verified origin).

---

Experiment

As I've found here, Google strips requests of certain headers¹, but not all². Let's find if there are such headers for Cloud Scheduler.

¹ E.g. you can't send any X-Google-* headers (found experimentally, read more)

² E.g. you can send X-Appengine-* headers (found experimentally)

Flask app used in the experiment:

@app.route('/echo_headers')
def echo_headers():
    headers = {h[0]: h[1] for h in request.headers}
    print(headers)
    return jsonify(headers)

Request headers sent by Cloud Scheduler

{
  "Host": []
  "X-Forwarded-For": "0.1.0.2, 169.254.1.1",
  "X-Forwarded-Proto": "http",
  "User-Agent": "AppEngine-Google; (+http://code.google.com/appengine)",
  "X-Appengine-Queuename": "__scheduler",
  "X-Appengine-Taskname": [private]
  "X-Appengine-Taskretrycount": "1",
  "X-Appengine-Taskexecutioncount": "0",
  "X-Appengine-Tasketa": [private]
  "X-Appengine-Taskpreviousresponse": "0",
  "X-Appengine-Taskretryreason": "",
  "X-Appengine-Country": "ZZ",
  "X-Cloud-Trace-Context": [private]
  "X-Appengine-Https": "off",
  "X-Appengine-User-Ip": [private]
  "X-Appengine-Api-Ticket": [private]
  "X-Appengine-Request-Log-Id": [private]
  "X-Appengine-Default-Version-Hostname": [private]
}

Proof that header X-Appengine-Queuename is stripped by GAE

Limitations

This method is most likely not supported by Google SLAs and Depreciation policies, since it's not documented. Also, I'm not sure if header cannot forged when the request source is within Google Cloud (maybe they're stripped at the outside layer). I've tested with an app in GAE, results may or may not vary for other deployment options. In short, use at your own risk.

Prynne

If you are using Cloud Scheduler with AppEngine HTTP target then the x-appengine-cron and similar headers any more ... it sets x-cloudscheduler but google doesn't prevent anyone else from setting these. This really sucks as there is no simple way to verify. Was using terraform but will go back to using cron.yaml so I get a bit more magic. Also using appengine headers for validation is documented cloud.google.com/appengine/docs/flexible/nodejs/… – Gerladina

The only flakey way to validate cloud scheduler source without using appengine cron.yaml is to maybe look at the source ip, but this could change with future versions 'x-appengine-user-ip': '0.1.0.2' – Gerladina

Answer 3

This header should work:

map (key: string, value: string)

HTTP request headers.

This map contains the header field names and values. Headers can be set when the job is created.

Cloud Scheduler sets some headers to default values:

User-Agent: By default, this header is "AppEngine-Google; (+http://code.google.com/appengine)". This header can be modified, but Cloud Scheduler will append "AppEngine-Google; (+http://code.google.com/appengine)" to the modified User-Agent. X-CloudScheduler: This header will be set to true. X-CloudScheduler-JobName: This header will contain the job name. X-CloudScheduler-ScheduleTime: For Cloud Scheduler jobs specified in the unix-cron format, this header will contain the job schedule time in RFC3339 UTC "Zulu" format. If the job has an body, Cloud Scheduler sets the following headers:

Content-Type: By default, the Content-Type header is set to "application/octet-stream". The default can be overridden by explictly setting Content-Type to a particular media type when the job is created. For example, Content-Type can be set to "application/json". Content-Length: This is computed by Cloud Scheduler. This value is output only. It cannot be changed. The headers below are output only. They cannot be set or overridden:

X-Google-: For Google internal use only. X-AppEngine-: For Google internal use only. In addition, some App Engine headers, which contain job-specific information, are also be sent to the job handler.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.

https://cloud.google.com/scheduler/docs/reference/rest/v1/projects.locations.jobs#appenginehttptarget

Immanuel