Node.js maxing out at 1000 concurrent connections

Question

We're benchmarking node performance using a simple Hello World node server on AWS (EC2).

No matter what size instance we use Node always appears to max out at 1000 concurrent connections (this is NOT 1000 per second, but 1000 it can handle at 1 time). Shortly after that the CPU spikes and node basically freezes.

Node v0.10.5

var http = require('http');
var server = http.createServer(function (req, res) {
    res.writeHead(200, {'Content-Type': 'text/plain'});
    res.end('loaderio-dec86f35bc8ba1b9b604db6c328864c1');
});
server.maxHeadersCount = 0;
server.listen(4000);

Node should be able to handle more than this correct? Any thoughts would be greatly appreciated.

Also the file descriptors (soft, hard, system) are set to 65096)

Cloak

how many concurrent connections did you finally get? And on what EC2 instance? – Canso

@Canso apologies didn't see this. we we're doing this to calibrate the potential to see if our app's concurrent connections made sense and then to determine the best instance size to use for the price during scale out since our app was processing 100's of millions of requests a day. It's been a long time (obviously) but I think we got up to 3k or 4k on a small (which was single core) and it went down a little per core as we scaled up the instance sizes. Of course ec2 has changed the specs on all the instances since then. – Cloak

Answer 1

Use the posix module to raise the limit on the number of file descriptors your process can use.

Install posix

npm install posix

Then in your code that runs when you launch your app...

var posix = require('posix');

// raise maximum number of open file descriptors to 10k,
// hard limit is left unchanged
posix.setrlimit('nofile', { soft: 10000 });

Dysart

thanks but the file descriptors have already been upped to 65096 – Cloak

we gave this a try and it worked! Daniel do you know why setting the ulimit via etc/security/limits.conf doesn't work but using posix does? – Cloak

Limits are applied on login. If you didn't log out of the shell and then back in after editing /etc/security/limits.conf then you would see the old limits. You can check current limits with ulimit -a – Dysart

Figured it out. We were starting node via monit & upstart. Upstart has separate nofile defaults (apparently regardless of the OS). So setting the Upstart config limit nofile values solved the problem. The reason Posix worked was since posix launches a new process it wasn't subject to Upstart limits – Cloak

@Cloak could you post the final solution you are using now with upstart and monit? I'm having the exact same problem and like to ad something to elastic beanstalk that new instances are automatically started with the correct configs. – Philippines

and is it possible to detect that node is frozen and restart the services or restart the ec2 instance? – Philippines

@manual checkout this bryanmarty.com/2012/02/10/setting-nofile-limit-upstart – Cloak

Answer 2

You've reached the default limit of file descriptors a process can use (1024). You can check the limit on the command line by running "ulimit -n". To change the limit, you need to edit /etc/security/limits.conf. Add the follow block:

* soft nofile 65535
* hard nofile 65535
root soft nofile 65535
root hard nofile 65535

"*" applies to all users except root. Limits for root must be added separately. There are soft and hard limit. Users are allowed to change their own limits up to the soft limit but not exceeding the hard limit.

Once the file has been edited, log out and back in again. Verify the change by running ulimit -n. Restart your Node process and you should be good to go.

There's also a system-wide file descriptor limit that can be increased with the following command:

sysctl -w fs.file-max=65535

Dwarfish

Thanks. We had already upped the file descriptors to 65096 before the tests (Both Soft, Hard and System Wide) – Cloak