NodeJS on multiple processors (PM2, Cluster, Recluster, Naught)

Question

I am investigating options for running node in a multi-core environment.

I'm trying to determine the best approach and so far I've seen these options

• Use built in cluster library to spin up works and respond to signals
• Use PM but, PM2 -i is listed as beta.
• Naught
• Recluster

Are there other alternatives? What are folks using in production?

Fredfreda

Answer 1

I've been using the default cluster library, and it works very well. I've had over 10,000 concurrents(multiple clusters on multiple servers) and it works very well.

It is suggested to use clusters with domain for error handling.

This is lifted straight from http://nodejs.org/api/domain.html I've mades some changes on how it spawns new clusters for each core of your machine. and got rid of if/else and added express.

var cluster = require('cluster'),
    http = require('http'),
    PORT = process.env.PORT || 1337,
    os = require('os'),
    server;

function forkClusters () {
    var cpuCount = os.cpus().length;
    // Create a worker for each CPU
    for (var i = 0; i < cpuCount ; i += 1) {
        cluster.fork();
    }
}

// Master Process
if (cluster.isMaster) {

    // You can also of course get a bit fancier about logging, and
    // implement whatever custom logic you need to prevent DoS
    // attacks and other bad behavior.
    //
    // See the options in the cluster documentation.
    //
    // The important thing is that the master does very little,
    // increasing our resilience to unexpected errors.

    forkClusters ()

    cluster.on('disconnect', function(worker) {
        console.error('disconnect!');
        cluster.fork();
    });

}
function handleError (d) {
    d.on('error', function(er) {
        console.error('error', er.stack);

        // Note: we're in dangerous territory!
        // By definition, something unexpected occurred,
        // which we probably didn't want.
        // Anything can happen now!Be very careful!

        try {
            // make sure we close down within 30 seconds
            var killtimer = setTimeout(function() {
                process.exit(1);
            }, 30000);
            // But don't keep the process open just for that!
            killtimer.unref();

            // stop taking new requests.
            server.close();

            // Let the master know we're dead.This will trigger a
            // 'disconnect' in the cluster master, and then it will fork
            // a new worker.
            cluster.worker.disconnect();
        } catch (er2) {
            // oh well, not much we can do at this point.
            console.error('Error sending 500!', er2.stack);
        }
    });
}
// child Process
if (cluster.isWorker) {
    // the worker
    //
    // This is where we put our bugs!

    var domain = require('domain');
    var express = require('express');
    var app = express();
    app.set('port', PORT);

    // See the cluster documentation for more details about using
    // worker processes to serve requests.How it works, caveats, etc.

    var d = domain.create();
    handleError(d);

    // Now run the handler function in the domain.
    //
    // put all code here. any code included outside of domain.run will not handle errors on the domain level, but will crash the app.
    //

    d.run(function() {
        // this is where we start our server
        server = http.createServer(app).listen(app.get('port'), function () {
            console.log('Cluster %s listening on port %s', cluster.worker.id, app.get('port'));
        });
    });
}

Dye

Will this automatically spawn a worker per core? Seems like it would be straight forward to migrate this same idea to express. – Fredfreda

Yes. this automatically spawns a new cluster per core. You then start your express listener in each else statement. – Dye

I think app.listen() returns the wrapped server so wouldn't this work server = app.listen() – Fredfreda

sorry, I did var server twice. the second one shouldn't be there. I'm removing it. – Dye

Let us continue this discussion in chat. – Dye

@BrianNoah If I have code inside the cluster.IsWorker block it executes 6 times (or however many cores I have) every time. For example, when console logging something it will console log it 6 times (I have 6 cores), is that normal? (See my latest question if you can) Like, for example I have a gameloop interval setup that runs every second. It runs 6 times every second....... not sure if that's worse or better than just once? – Middlings

@NiCkNewman That means that it's every second per cluster. This might or might not be what you want. If you want, you can run this function inside of master, and use a PUB/SUB to delegate the events to the worker clusters. I'm not entirely sure what the question is though. I would suggest asking a new S/O question and give me the link. – Dye

@BrianNoah Hey Brian, no problems. Made the question! – Middlings

Answer 2

We use Supervisor to manage our Node.JS process's, to start them upon boot, and to act as a watchdog in case the process's crash.

We use Nginx as a reverse-proxy to load balance traffic between the process's that listen to different ports

this way each process is isolated from the other.

for example: Nginx listens on port 80 and forwards traffic to ports 8000-8003

Chintz

How are you handling graceful shutdown? – Fredfreda

We change the health check to return 404 while we finish handling the remaining requests and then ELB stops sending traffic to this server, and all the previous requests are handles before shutting down the server. – Chintz

1) How do you know when it is safe to actually shutdown 2) How are you handling uncaught exceptions? – Fredfreda

Answer 3

I was using PM2 for quite a while, but their pricing is expensive for my needs as I'm having my own analytics environment and I don't require support, so I decided to experiment alternatives. For my case, just forever made the trick, very simple one actually:

forever -m 5 app.js

Another useful example is

forever start app.js -p 8080

Callosity