How are concurrent requests handled by Nodejs express http server?

Question

I am building a Node.js application and wanted to understand how concurrent requests are handled.

I build a test server, where high CPU load is being simulated by waiting 10 seconds. To test the behavior, I open two browser tabs and refresh the page simultaneously.

const http      = require('http');
const express       = require('express');
const bodyParser        = require('body-parser');
const app       = express();
const server        = require('http').createServer(app);  

app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }));

app.get('*', function (req, res, next) {
    var requestTime = new Date().getTime(),
            executionTime;

    doHeavyWork(requestTime, function(error){
        res.send({
            requestTime     : requestTime,
            executionTime   :   executionTime
        });
    });

});

function doHeavyWork (requestTime, callback) {
    var sleepSeconds = 10;

    while (requestTime + sleepSeconds*1000 >= new Date().getTime()) {}

    callback(null);
}

server.listen(1337, '127.0.0.1');

From what I heard about Node.js, I was expecting both Tabs to finish loading in the same time. In reality the Tab which is refreshed first also finishes first. The next tab loads after additional 10 seconds. So basically, the server processes the requests one at a time instead of processing them simultaneously. What am I missing here?

Fitzgerald

The problem is with how you implemented your "CPU Load" simulation; you are blocking the event loop by using a while loop. Instead, why not use a setTimeout that responds asynchronously to the requesting client. Node really relies on the developer not blocking the event loop, since it is single-threaded. Anything that blocks the event loop will prevent any other operation from happening, until the current operation finishes. Try to use asynchronous methods as much as possible! – Hirsute

Answer 1

To answer your question without getting into the nitty gritty of how Node works (which I advise you read), the behaviour you see is exactly what I'd expect to see based on your code.

For each Node instance running there is a single processing thread, in high-volume scenarios it's recommended to do as little CPU-bound operations as possible as to not block that thread. In your example, each request is running a 10s CPU-bound operation which means Node can't process any new requests until that request completes.

If you want to better demonstrate the throughput of Node, use a non-blocking example e.g. use of a timer

app.get('*', function (req, res, next) {
  var requestTime = new Date().getTime(),
        executionTime;

  setTimeout(() => {
    res.send({
      requestTime,
      executionTime: new Date().getTime()
    });
  }, 10000);
});

Claytor

You can checkout a demo here at: express-multiclient-demo-1bfoxa1iul24.runkit.sh (wait for 10s to resolve timeout) source code here at: runkit.com/humble-barnacle001/express-multiclient-demo/v2.0.0 – Rawley