nodejs single thread? how does it works if there are lots of concurrent requests

Question

I am new to nodejs, currently learning it. I know nodejs is single-threaded, however I am wondering: if I use non-blocking method, for example, there are 20000 concurrent requests, and there is one request need to take a long time, other threads takes shorter time.

So once we met the longer time request, nodejs will say "hi, please pause, other request need to use", is there a default time for nodejs to detect whether this need to be paused? Correct me if I use the wrong word "puase" (because I think since it is single-thread, so it should stop computing for one request, if another request wants to be processed).

In the situation I assumed, then the longer request need lots of time that it can be processed?

Bunyabunya

If the one that takes a long time does a lot of synchronous things, yes, you'll have a problem, but if it's asynchronous, such as reading a file or writing a file, other requests will continue to be processed while waiting for the asynchronous action to complete. javascript runs in an event loop. Because of this, you should avoid using synchronous actions on a node.js application that is serving requests. – Cotten

@KevinB yes, i understand others will be processed if i use asynchonous. But my question is that since it is single-thread, if you process other request takes shorter time, during that time the longer-time request is not processed during that time. Correct me if i am wrong. Then there are lots of shorter time request which will cause the longer time request not precessed. And need more time to finish the longer request. Is that right? – Bunyabunya

Here's an analogy. Node.js is a King. He sits on his throne, and servants come in with requests looking for responses. The king only hears one request at a time, and only responds once at a time. However, he has many servants to do his work. Servant A comes in immediately followed by servant B. the king listens to servant A, and then sends him to get coffee. He then listens to servant B and sends him to open the window. If servant B gets back before servant A, servant B will be dismissed first, while the king is still waiting on servant A. When servant A gets back, the king will dismiss him 2. – Cotten

Many requests can be getting processed at the same time, but only one will be received or responded to at a time. – Cotten

@KevinB I see what you mean, the single thread is for the I/o. But not for inner processing. The analogy is great. Thanks – Bunyabunya

Why the down votes? The fact that the OP doesn't understand how node.js work is not enough to down vote. It looks like a valid question to me. +1 – Goodrow

valid to me as well, +1 to @Bunyabunya keep it up! – Stultz

See also: What exactly is a Node.js event loop tick – Prithee

Answer 1

It is true that node.js can only receive or respond to a single request at a time, however, it can process multiple requests at the same time.

For example, lets say you have a node.js application that is a rest api, and every request results in a call to the database except for endpoint C. The rest api has 3 endpoints.

Endpoint A: takes 3 seconds to talk to the database

Endpoint B: takes 2 seconds to talk to the database

Endpoint C: does not talk to the database, it just returns static text.

It isn't possible for more than 1 request to happen at the same time. One is always first, regardless of how close the timestamps are.

Now, lets say we have 10 requests happening at the same time, in this order:

ABCBCCAABC

Here's how they will be received and responded to:

REC:A
REC:B
REC:C
RESP:C
REC:B
REC:C
RESP:C
REC:C
RESP:C
REC:A
REC:A
REC:B
REC:C
RESP:C
// 2 seconds later
RESP:B
RESP:B
RESP:B
// 1 second later
RESP:A
RESP:A
RESP:A

C always happens immediately because it doesn't do any asynchronous actions. You then receive all of B, and then all of A because B is faster than A.

The order of B and A may differ though based on database setup, for example, if it's set to only handle x queries at a time.

If we introduced and endpoint D that performs a synchronous action that takes 6 seconds, you'll get this result:

ADABC

REQ:A
REQ:D
// 6 long seconds later...
RESP:D
REQ:A
REQ:B
REQ:C
RESP:C
// 2 seconds later...
RESP:B
// 1 second later...
RESP:A
RESP:A

because D stops all javascript processing while it's synchronous action takes place. 1 endpoint that performs a synchronous action is an easy point of failure for someone to bring your api to a halt.

Cotten

Great. I see. But I have a question. What's the advantage over other language's "use a new thread once a new request comes in" I see the difference is in I/O part. But what's the advantage – Bunyabunya

That i don't think i can really answer, and i doubt it's a "cut and dry" answer anyway, as each method has it's own advantages/disadvantages. – Cotten

yes, you are right. But i see lots of people say it will avoid race condition. avoid spending time on deadlocks. But still i think there might be race conditions inside, since inner processing for each request is still concurrent. – Bunyabunya

there certainly can be race conditions if you don't code properly. From what little experience i have with node.js apis compared to, say, coldfusion, node.js apis are far more verbose, i have to actually code a lot more of the pieces to make it all work (of course, there's modules that do this for you too such as express and loopback.) – Cotten

Also keep in mind that just because node.js is single-threaded doesn't mean you can't use multiple threads for a single app. For example, i run my api in a cluster of 4, one per cpu. The requests get split among the 4 apps automatically, and since i'm using a db table for session management, all 4 can access session data. If your database is on a separate server, you can then easily add more servers (behind a loadbalancer) running more clusters without changing anything. – Cotten

@KevinB wow, yes, that's right. I just started learning nodejs , it is really verbose compare to others. Also you example of different clusters is a good example. – Bunyabunya

Answer 2

I was having the same problem, you can use the cluster and it can solve the problem to some extent. Here is a demo code:

import cluster from 'cluster';
import os from 'os';
import http from 'http';

if (cluster.isPrimary) {
  // Primary process
  const numCPUs = os.cpus().length;
  console.log(`Primary ${process.pid} is running`);

  // Fork workers
  for (let i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  cluster.on('exit', (worker, code, signal) => {
    console.log(`Worker ${worker.process.pid} died`);
    cluster.fork(); // Restart a new worker if one dies
  });
} else {
  // Worker processes
  http.createServer((req, res) => {
    res.writeHead(200);
    res.end('Hello, world!\n');
  }).listen(8000);

  console.log(`Worker ${process.pid} started`);
}

Classicist