libuv uses blocking file system calls internally – Why? How?

Question

I just learned that Node.js crown jewel libuv uses blocking system calls for file operations. The asynchronous behavior is implemented with threads! That raises two questions (I only care about Unix):

1. Why is it not using the non-blocking filesystem calls like it does for networking?
2. If there are one million outstanding file reads, it probably does not launch one million threads... What does libuv do??

Grande

Non-blocking I/O may not be possible always or be of significant use. Read remlab.net/op/nonblock.shtml – Tilefish

Answer 1

1. The same non-blocking API can not be used, as O_NONBLOCK and friends don’t work on regular files! For Linux AIO is available, but it has it’s own quirks (i.e. depends on the filesystem, turns silently blocking for some operations).

2. I have no idea.

Grande

Yeah, O_NONBLOCK just doesn't work on files with linux so uv simulates the non blocking behaviour with a thread pool. – Scepter

So because O_NONBLOCK not working on linux, windows does not have the benefit to Non-blocking file APIs when using Node? How about .Net Core? – Bromley

Answer 2

1. Most likely to support synchronous operations such as fs.renameSync() vs fs.rename().

2. It uses a thread pool, as explained in the "Note" at the link you provded.

   [...] but invoke these functions in a thread pool and notify watchers registered with the event loop when application interaction is required.

   So, it creates a limited number of threads and reuses them as they become available.

Also, regarding the quip of "crown jewel:" Node.js and libuv aren't magic. They're good tools to have at your disposal, but certainly have their limitations.

Though, the hyperbole of "one million file reads" would be a stretch for any platform to manage without constraint.

Dow

1) I didn’t meant to include rename() operations. I mean file descriptor based operations (i.e. what the kernel can do non-blocking). 2) So libuv doesn’t handle all operations at once. It just takes the first bunch to put it the thread pool and keeps the rest in a FIFO buffer? – Grande