Google App Engine and dos.xml

Question

My understanding is that the only way to really mitigate a DDoS attack is to automate the process of blacklisting IP addresses/ranges.

Google App Engine (GAE) allows you to configure and upload a dos.xml file and specify IP addresses/ranges to blacklist at any given time.

Obviously, if my web app is under a well-orchestrated DDoS attack, the IP addresses/ranges that will be attacking me will be constantly changing.

How often does GAE allow me to update dos.xml? How long does it take the changes to go into effect? I ask because I'm devising an AutoBlacklister system that inspects the IP addresses that it believes to be the attackers, and will update dos.xml dynamically. If there are more than 100 attackers (GAE restricts you to 100 address/ranges) then only the Top 100 "worst offenders" will be on the list.

But, if dos.xml can only be updated with a certain periodicity (like once a day, etc.), and if it takes too long (more than a few minutes!) to take effect,then this system is pretty much useless against a real DDoS.

Also, this question assumes there's a way to automate the uploading of dos.xml: is there? I would imagine there's a secure URL I could upload the file to with something like HttpClient, but with GAE, you never know what terms/restrictions you're going to face! Thanks in advance!

Salta

Not entirely related, but just to potentially save some troubleshooting later on, the documentation says that it's dos.xml, rather than ddos.xml. – Catarrhine

Thanks for pointing that out (+1) - OP is now updated. – Salta

FWIW, the newly released GAE firewall supports programmatic updates of the firewall rules via the (REST) Admin API: cloud.google.com/appengine/docs/admin-api/reference/rest/v1beta/… – Teenateenage

Answer 1

Blacklisting IPs is not 100% DDoS proof mitigation techniques as:

A.) Botnet DDoS will use legit IPs (i.e. Trojan Botnet) and, in this case, IP blockage will also prevent access from legitimate users.

B.) This will do nothing at all against network DDoS attack (i.e. SYN Flood) - an attack which uses spoofed IPs and does not even need to establish a full 2-way connection for the DDoS to work. (To stop this you`ll need to have some kind of front-gate reverse proxy in place, to prevent access until full 2-connection is established --> ACK received.)

For full DDoS protection you`ll need to have a large-enough "pipe", either by investing in hardware (too expansive and thus usually not cost-effective) or in front-gate proxy solution that will balance the extra traffic while allowing you to stay fully operational (i.e. Cloud proxy).

Gass

Does Google take care of any of this to prevent the requests getting to the app in the first place? – Botts

Not 100% sure. I know they allow updates but not familiar with their internal prevention policies. Would like to learn more about it myself. – Gass

Answer 2

You can update dos.xml through AppCfg. It is possible to update this file without a full redeployment of the server, which is an expensive process. As far as I'm aware, there is no limit on how often this update can be performed.

Full deployment does have a limit which is described here:

The number of times the application has been uploaded by a developer. The current quota is 1,000 per day.

Edveh