How does Google App Engine Autoscaling work?

Question

This question is about Google App Engine quotas and instances.

I deployed a GAE app without specifying any specific scaling algo. From their docs, it seems like the default is auto-scaling.

So when do they scale the app to another instance, i.e. when exactly does a new instance spawn? What request/s causes the second instance to get started and traffic to be split?

Photogravure

do more research before assuming you are being ripped off. not funny. and very well explained in their docs. – Rhodos

And yet there is no answer to how many requests/s each instance class can serve before a new instance is spawned. Of course what their docs explain is logical: the instance serves requests until it no longer can serve them, and then the new requests are sent to a new spun instance. But don't you think providing a number to that will help? Their docs are huge, so forgive me if I missed something, but I wouldn't have made any bold statements in my question had I found the answer after enough searching – Photogravure

As an example, this is what Heroku's docs say about dynos: devcenter.heroku.com/articles/dynos#dynos-and-requests. At least the user knows they're good up until about 1000 requests a second (of course taking into account their latency). – Photogravure

Answer 1

Actually it is fairly well explained.

From Scaling dynamic instances:

The App Engine scheduler decides whether to serve each new request with an existing instance (either one that is idle or accepts concurrent requests), put the request in a pending request queue, or start a new instance for that request. The decision takes into account the number of available instances, how quickly your application has been serving requests (its latency), and how long it takes to spin up a new instance.

Each instance has its own queue for incoming requests. App Engine monitors the number of requests waiting in each instance's queue. If App Engine detects that queues for an application are getting too long due to increased load, it automatically creates a new instance of the application to handle that load.

It is possible to tune the parameters for autoscaling, see Change auto scaling performance settings and Scaling elements

If you really want to put a hard limit on the number of instances you'll probably have to use basic scaling with a max_instances config or manual scaling.

Bullace