What happens when a single request takes a long time with these non-blocking I/O servers?

Question

With Node.js, or eventlet or any other non-blocking server, what happens when a given request takes long, does it then block all other requests?

Example, a request comes in, and takes 200ms to compute, this will block other requests since e.g. nodejs uses a single thread.

Meaning your 15K per second will go down substantially because of the actual time it takes to compute the response for a given request.

But this just seems wrong to me, so I'm asking what really happens as I can't imagine that is how things work.

Cort

Answer 1

Whether or not it "blocks" is dependent on your definition of "block". Typically block means that your CPU is essentially idle, but the current thread isn't able to do anything with it because it is waiting for I/O or the like. That sort of thing doesn't tend to happen in node.js unless you use the non-recommended synchronous I/O functions. Instead, functions return quickly, and when the I/O task they started complete, your callback gets called and you take it from there. In the interim, other requests can be processed.

If you are doing something computation-heavy in node, nothing else is going to be able to use the CPU until it is done, but for a very different reason: the CPU is actually busy. Typically this is not what people mean when they say "blocking", instead, it's just a long computation.

200ms is a long time for something to take if it doesn't involve I/O and is purely doing computation. That's probably not the sort of thing you should be doing in node, to be honest. A solution more in the spirit of node would be to have that sort of number crunching happen in another (non-javascript) program that is called by node, and that calls your callback when complete. Assuming you have a multi-core machine (or the other program is running on a different machine), node can continue to respond to requests while the other program crunches away.

There are cases where a cluster (as others have mentioned) might help, but I doubt yours is really one of those. Clusters really are made for when you have lots and lots of little requests that together are more than a single core of the CPU can handle, not for the case where you have single requests that take hundreds of milliseconds each.

Stirrup

Answer 2

Everything in node.js runs in parallel internally. However, your own code runs strictly serially. If you sleep for a second in node.js, the server sleeps for a second. It's not suitable for requests that require a lot of computation. I/O is parallel, and your code does I/O through callbacks (so your code is not running while waiting for the I/O).

On most modern platforms, node.js does us threads for I/O. It uses libev, which uses threads where that works best on the platform.

Nitramine

There's a great explanation here which is basically what you've said, but explains a little more. – Persist

Answer 3

You are exactly correct. Nodejs developers must be aware of that or their applications will be completely non-performant, if long running code is not asynchronous.

Everything that is going to take a 'long time' needs to be done asynchronously.

Autotrophic

Answer 4

This is basically true, at least if you don't use the new cluster feature that balances incoming connections between multiple, automatically spawned workers. However, if you do use it, most other requests will still complete quickly.

Edit: Workers are processes.

Crummy

Answer 5

You can think of the event loop as 10 people waiting in line to pay their bills. If somebody is taking too much time to pay his bill (thus blocking the event loop), the other people will just have to hang around waiting for their turn to come.. and waiting...

In other words:

Since the event loop is running on a single thread, it is very important that we do not block it’s execution by doing heavy computations in callback functions or synchronous I/O. Going over a large collection of values/objects or performing time-consuming computations in a callback function prevents the event loop from further processing other events in the queue.

Marlea

Answer 6

Here is some code to actually see the blocking / non-blocking in action:

• With this example (long CPU-computing task, non I/O):

  var net = require('net');
  handler = function(req, res) {
      console.log('hello');
      for (i = 0; i < 10000000000; i++) { a = i + 5;  }
  }
  net.createServer(handler).listen(80);
  

  if you do 2 requests in the browser, only a single hello will be displayed in the server console, meaning that the second request cannot be processed because the first one blocks the Node.js thread.

• If we do an I/O task instead (write 2 GB of data on disk, it took a few seconds during my test, even on a SSD):

  http = require('http');
  fs = require('fs');
  buffer = Buffer.alloc(2*1000*1000*1000);
  first = true;
  done = false;
  
  write = function() {
      fs.writeFile('big.bin', buffer, function() { done = true; });
  }
  
  handler = function(req, res) {
      if (first) {
          first = false;
          res.end('Starting write..')
          write();      
          return;
      }
      if (done) { 
          res.end("write done."); 
      } else {  
          res.end('writing ongoing.'); 
      }
  }
  
  http.createServer(handler).listen(80);
  

  here we can see that the a-few-second-long-IO-writing-task write is non-blocking: if you do other requests in the meantime, you will see writing ongoing.! This confirms the well-known non-blocking-for-IO features of Node.js.

Osage