Google App Engine firewall and internal access

Question

I have two services running on Google App Engine (flex, same project), and I'd like one service to call the other using HTTPS.

On top of this, I've setup the firewall, only allowing 0.1.0.40 and 10.0.0.1.

I'm also setting the X-Appengine-Inbound-Appid header in the request.

Unfortunately, I'm getting a "403 Access is forbidden" error (which disappears when disabling the firewall).

Right now, I am using the xxx.appspot.com URL to call the service. Should I use some internal URL instead? It seems that the request is seen as external by the firewall.

Thank you!

Ancipital

Can I ask you in which language are you doing this? Also, are you using URL Fetch for the communication? – Samba

Javascript, as I'm using Node with apollo-server. As for the communication, it's using native apollo-server functions. – Ancipital

And are you using the correct port? It should be 8080. – Samba

Yes, as disabling the firewall works. Also, it's not 8080 but 443 (HTTPS). – Ancipital

Do you see the X-Appengine-Inbound-Appid request header on the receiving side? – Lazuli

No... it's probably stripped out by the firewall? Maybe because this is a flex environment? – Ancipital

Answer 1

Our team had a similar issue. We are denying all outside access except our other App Engine Services, some of which reside in different GCP projects. The only way to allow access from your other App Engine services through the firewall is to have the consuming service use the URL Fetch Service and pass in the appspot.com URL as you mentioned. You also would set the followRedirects to false.

But, in your situation this won't work. According to this: https://cloud.google.com/appengine/docs/flexible/nodejs/glossary It's only available for Java, Python, PHP & Go. In these cases, you would simply add a Whitelist rule for 0.1.0.40 and 10.0.0.1.

If you don't want to create a proxy service in Google Compute Engine (as mentioned here), you will have to add in a bunch of very large CIDR ranges as specified here: https://cloud.google.com/appengine/kb/

Hypertension

Answer 2

On the page that tells you how to allow requests from a differnt App Engine service it's said that IPs, 0.1.0.40 and 10.0.0.1 are the ones that you must consider, but NOT the ONLY ones:

To control the access of requests from other App Engine apps or services, you might need to create rules to accommodate the IP addresses that are used for service-to-service communication. If your app communicates with other apps or services in App Engine, you must consider how to handle requests from the following IP addresses: ... (shortly: 0.1.0.40 and 10.0.0.1)

In order to allow incoming request from different Flexible services you can read this answer.

Regarding X-Appengine-Inbound-Appid headers they are automatically set by App Engine Standard services when they are making a request to a different Google Standard or Flexible service, but it can't be added by your application because Google will strip them automatically when you set them on your own or when requests is coming from outside of GCP for security reasons, according to this, this and this.

Setup

To see what is happening I created two App Engine Flex services, A and B. Cron would send a request to A, which then would send a request to B. Both A and B would print out headers for all interactions between them. Those print-outs can be read in Stackdriver Logging.

Observations

When The Default Firewall Rule is Set to Allow

• Cron requests to A have the following headers

X-Appengine-Cron: true
X-Appengine-Queuename: __cron
X-Forwarded-For: 10.0.0.1, 10.0.0.1

• The answer from B to A had the following header:

X-Forwarded-For: xxx.xxx.xxx.xxx, yyy.yyy.yyy.yyy

When The Default Firewall Rule is Set to Deny

• It turned out that if 10.0.0.1 is blocked then cron jobs fail, which is contrary to what this guide is saying:

Task Queues and Cron traffic will be allowed by the firewall, even when the default rule is set to deny.

• The request from A only reached B when allowing xxx.xxx.xxx.xxx, which is an IP from the Flexible instance, is allowed in the App Engine Firewall.

Reest

I'm having the same question from OP. I deny all IP (*) and whitelist all internal app engine IP (10.1.0.41, 0.1.0.40, 10.0.0.1 and 0.1.0.30). Yet I'm having a 403 error also. What is this censored ip xxx.xxx.xxx.xxx ? What else can I do? – Branscum

Answer 3

Task Queue method

You can use task queue to talk between the services. Let's consider these two services in the same project

service 1: https://service1.appspot.com

service 2: https://service2.appspot.com/recieve-task-endpoint

Flow:

service1 -> create task [service2 endpoint] -> cloud task queue -> service2 receives as [GET] request

App engine services can talk internally with cloud task and cloud scheduler despite the firewall rules.

To push the task to the queue Refer this code for Python. Here you can specify the public url of the service 2 https://service2.appspot.com/recieve-task-endpoint.

Peyter