Config of nginx to filter http flood

Question

A have a http flood on my server, not so much queries, but anyway. Queries in log

95.55.237.3 - - [06/Sep/2012:14:38:23 +0400] "GET / HTTP/1.0" 200 35551 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US)" "-" | "-" 93.78.44.25 - - [06/Sep/2012:14:38:23 +0400] "GET / HTTP/1.0" 200 36051 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US)" "-" | "-" 46.118.112.3 - - [06/Sep/2012:14:38:23 +0400] "GET / HTTP/1.0" 200 35551 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US)" "-" | "-"

I tried this filters in nginx config

server {
    .....
    set $add 1;
    set $ban '';

###### Rule 1 ########
if ($http_referer = '-' ) {
    set $ban $ban$add;
}
if ($request_uri = '/') {
    set $ban $ban$add;
}

if ($http_user_agent = 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US)') {
    set $ban $ban$add;
}

if ($ban = 111) {
    return 444;
}
######################
......
}

but stil bot queries get 200 OK. Could somebody help?

Nirvana

Answer 1

Try adding something like the following directives to your config to prevent http flooding:

http {
  limit_conn_zone $binary_remote_addr zone=conn_limit_per_ip:10m;
  limit_req_zone $binary_remote_addr zone=req_limit_per_ip:10m rate=5r/s;

  server {
    limit_conn conn_limit_per_ip 10;
    limit_req zone=req_limit_per_ip burst=10 nodelay;
  }
} 

See http://nginx.org/en/docs/http/ngx_http_limit_conn_module.html and http://nginx.org/en/docs/http/ngx_http_limit_req_module.html for more info

There's all the following directive http://nginx.org/en/docs/http/ngx_http_core_module.html#limit_rate

NOTE: http://www.botsvsbrowsers.com/details/504401/index.html says the above user agent is not a known bot

Multitudinous

this can be bad for many computers on the same lan, sharing one ip address – Carny

@Carny seriously how many people browse from the same lan nowadays if not for multiplayer game playing. you can even adjust the amount of ips allowed in the time you set... – Pasty

@stupidtroll do you know routers? they are used in every home and enterprises, putting all users under one or few internet IPs – Carny

an HTTP flood typically sends faaaaaaar more requests per second than a typical large office with a group of web users sitting behind a router; it's just a matter of tweaking your rate and configuring burst. – Arie

Answer 2

You can also block specific IP, as additional measure.

http{
  deny 127.45.4.1;
  ...
}

Or put blocked IPs in separate file

http{
  include blockedips.conf
  ...
}

blockedips.conf

deny 1.12.4.5;

Macpherson

Answer 3

You could also block specific country

http{
   geoip_country /usr/share/GeoIP/GeoIP.dat;
    map $geoip_country_code $allowed_country {
        default yes;
        FK no;
        FM no;
        EH no;
    }
}

GeoIP.dat can be downloaded from http://dev.maxmind.com/geoip/geoip2/geolite2/ (I am not affiliated with maxmind)

Macpherson