Optimize Nginx + PHP-FPM for faster response times (for Openx adserving)

Question

I'm currently running Nginx + PHP-FPM for serving ads on OpenX. Currently my response times are horrible, even during times of low load. However, my CPU and Memory resources are fine, so I can't seem to figure out what the bottleneck is.

My current config for nginx and php-fpm is:

worker_processes 20;
worker_rlimit_nofile 50000;

error_log  /var/log/nginx/error.log;
pid        /var/run/nginx.pid;

events {
    worker_connections  15000;
    multi_accept off;
    use epoll;
}

http {
    include       /etc/nginx/mime.types;

    access_log  /var/log/nginx/access.log;

    sendfile        on;
    tcp_nopush     off;

    keepalive_timeout  0;
    #keepalive_timeout  65;
    tcp_nodelay        on;

    gzip  on;
    gzip_disable "MSIE [1-6]\.(?!.*SV1)";
    gzip_comp_level 2;
    gzip_proxied    any;
    gzip_types    text/plain text/html text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

server {
    listen   80;
    server_name  localhost;
    access_log  /var/log/nginx/localhost.access.log;

# Default location
    location / {
        root   /var/www;
        index  index.php;
    }

## Parse all .php file in the /var/www directory
    location ~ .php$ {
        fastcgi_pass   localhost:9000;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME  /var/www$fastcgi_script_name;
        include fastcgi_params;
        fastcgi_param  QUERY_STRING     $query_string;
        fastcgi_param  REQUEST_METHOD   $request_method;
        fastcgi_param  CONTENT_TYPE     $content_type;
        fastcgi_param  CONTENT_LENGTH   $content_length;
        fastcgi_ignore_client_abort     off;
    }

PHP-FPM:
rlimit_files = 50000
max_children = 500

I only included the PHP-FPM paramaters I've changed for PHP-FPM.

Does anyone have any tips on how I can optimize it so I can serve more requests? I'm seeing horrendous response times right now.

Octet

If you want a faster response time with PHP, you should use Apache. Nginx+php-fpm is good for high traffic, but for 90% of web sites, it will be slower than Apache+mod_php ^^ – Miles

Answer 1

First off, way too many workers, and limits set excessively high. The max worker count for php-fpm alone would bog your server down quite a bit. Uncapping the limits on a server won't necessarily speed it up but may actually have the opposite effect.

1. Worker Count: 20 makes little sense if you do not have a 20 processor/core machine, you're actually causing a negative effect as the workers will have excessive content swapping. If you're running a dual core processor, 2 workers should suffice.

2. Worker Connections: Again, just throwing a limit into the heavens doesn't solve your problems. If your ulimit -n output is something like 1024, then your worker connections would need to be set to 1024 or less (maybe even 768), its unlikely that you'll have 2 x 1024 simultaneous connections especially with something like PHP.

3. Root location, and PHP settings, refer to http://wiki.nginx.org/Pitfalls , it works best if you put your root directive at the server {} level, not the location level. Once you do that you can use $document_root$fastcgi_script_name as the SCRIPT_FILENAME value as $document_root will be automatically propagated to location blocks below it.

4. You may wish to handle static files directly, in other words:

   location ~* \.(ico|css|js|gif|jpe?g|png)$ {
       expires max;
       add_header Pragma public;
       add_header Cache-Control "public, must-revalidate, proxy-revalidate";
   }
   

5. Use a PHP Accelerator, namely APC (with apc.enabled=1 in php.ini) or XCache, and be mindful of your php settings, such as the memory_limit. For example if you only have a system with 2GB of rams, it makes very little sense to allow 500 workers with a limit of 128MB each. Especially true if you're also running other services on your server.

Cartomancy

Answer 2

Would be also useful to put:

access_log off;

As i suppose you don't really care of log generation on these requests.

Resemblance

Answer 3

You should definitely reduce the number of workers as I doubt you have 20 cores/processors. Additionally, I'd look into your database server, there's a big possibility that the problem is there.

Additionally you've upped the worker_rlimit_nofile to 50000, I hope you know that operating system usually set the limit to 1024 (default), you can try to request what's the current limit by typing ulimit -n

You can raise hard limit of NOFILE (number of open files) by executing this command ulimit -n 50000 in init.d or visit this other question on stackoverflow to learn how to use limits.conf to permanently set limits system wide.

Jugum

Answer 4

Really pushing performance to the max with nginx and php5-fpm is an art. It takes really understanding the kind of contents you are serving.

For example, I don't see any try_files usage, or any kind of caching in your configuration. Do you know nginx comes with built-in memcache support? You can cache images and html/css, as well as php pages. If you care mostly for clicks, those clicks will still be counted even if displays are not.

Put your banners in a tmpfs mount, don't log access_log or error_log, disable modules you don't need in php, use a recent version of mysql, use innodb to reduce table locking, play with the flushing method of innodb to reduce disk writes, increase maximum memory tables in mysql to reduce the creation of temporary files on disk when joins are requested via SQL, etc.

Nginx is just one part of a very large and complex formula. I have not even mentioned Kernel params to optimize the TCP Stack and Network Card performance, Swap usage, Memory usage, or gzip compression of HTML/CSS you may be serving via OpenX (If you are).

And yes, like others above me mentioned, your settings look excessive and show a lack of understanding of basic optimization concepts. In other words, hire a professional :-)

Mushro

Answer 5

do you have 20 processors or cores on your machine? also maybe try events with the default for your OS... maybe more fcgi processes instead of more nginx... probably starting with 2 - 4 nginx workers is enough...

Luht

Answer 6

Definitely way too may workers as folks have mentioned. I personally prefer xcache over APC for php opcode caching. You should check out configuration in modified centmin auto bash shell nginx/php-fpm install script http://vbtechsupport.com/796/

Fadil

Answer 7

The most effective way to make a server system much faster is to use Facebooks HipHop Virtual Machine (HHVM) instead of PHP (PHP not must be installed any more).

HHVM uses upstream of the CPU a "Just in Time Compiler" and executes the usually PHP code 5 to 10 times faster than PHP itself, it makes it possible to get along with a smaller number of servers or smaller servers and to reduce the power consumption essentially. Wikipedia used HHVM to reduce the CPU server load by the factor 5: http://www.golem.de/news/php-facebooks-hhvm-macht-wikipedia-schneller-1501-111515.html

It can be installed together with Nginx as Linux package and be included in Nginx very easily similar as FastCGI, and soon after a few minutes it can be tested trough a small "Hello World" PHP file: https://github.com/facebook/hhvm/wiki/Getting-Started

The new PHP7 PHPNG should be in truth according to benchmark tests only 30% faster.

thanks for upvoting

Impotence