"readable" event occurs twice

Question

var fs = require('fs');

var file = fs.createReadStream('./zeros.txt');
var dataSize = 0;

file.on('readable', function () {
    var data = file.read(10);
    console.log('readable size = ', data.length);
    console.log(data.toString());
});

Thie file "zeros.txt" contains 700 characters "0"

As i understand, after calling read(10), stream must stop and wait for a new read() call. However, the result of calling:

readable size =  10
0000000000
readable size =  10
0000000000

Quagmire

Answer 1

After Node.js will load the file (whole or just a part, depends on the size of the file itself) into the buffer (using the push() method) it will emit readable event to indicate that some data has been read into the buffer and is ready to use. Then after you call read(10), you will free the buffer and then Node.js will fill the buffer automatically again and emit the readable event again to indicate that there is still some data to read from the buffer. If you would call read(700), no next readable event would be emitted again.

Flowing- and Non-Flowing Mode

Unlike when listening for a data event the stream will stay in so-called non flowing mode. That means that the developer will be responsible for freeing the stream (reading from the stream). On the other hand when listening for a data event the stream will autmatically enter the so-called flowing mode which means that the stream itself will be responsible for freeing itself, i.e stream will fill and empty itself till he underlying system (in this case the zero.txt will be fully read). Note the buffer will be filled with data automatically in either mode.

Flowing mode

An example of the non-flowing mode, where we have to empty the buffer manually (using the read() method):

var fs = require('fs'),
util = require('util');

// I have increased the file size to 19 MB (about 19 mln characters);
// Cause of the buffer capicity.
var file = fs.createReadStream('./zeros.txt'); 
var dataSize = 0;

// Readable will be called when the buffer has been filled with data.
// Initially Node.js will fill the buffer with data automatically,
// so this event will be called automatically aswell of course.
// Once the buffer will be free again after the first fill, Node.js
// will fill the buffer automatically again. Node.js just watches this stream
// and makes sure to fill it, when there is still some unread data in the zero.txt file.
file.on('readable', function() {
var i = 0; // we will count how many times did while loop, for fun

// If the buffer will be empty Node will write data to the buffer
// automatically, we don't have to care about that. However
// you can specify the buffer capicty manually if you want.
console.log('loading more data from the underlying system');

// This will make the stream read 1000 bytes
// it will also return a value NULL if there is not enough 
// data to read from the buffer (meaning buffer has been fully read 
// or there is still some data but you are trying to read 1000 bytes 
// and there is less than 1000 bytes left)
while(file.read(1000) !== null) {
    i++;
}
// At this moment while loop has read everything from the buffer.
// The buffer is now empty. After this comment console.log will execute
// Node.js will fill the buffer again with new data automatically.
// And then the 'readable' event will fire again.
console.log("had to loop: " + i + " times before the buffer was empty");
})

Last few results from the console:

loading more data from the underlying system
had to loop: 66 times before the buffer was empty
loading more data from the underlying system
had to loop: 65 times before the buffer was empty
loading more data from the underlying system
had to loop: 66 times before the buffer was empty
loading more data from the underlying system
had to loop: 46 times before the buffer was empty
loading more data from the underlying system
had to loop: 1 times before the buffer was empty

Non-Flowing mode

That was the non-flowing mode, since we had to free the buffer manually. Now we will proceed to the flowing mode. Setting a data event listener on a Readable Stream switches the stream to flowing mode from the initial non-flowing mode. That means that the buffer will be emptied automatically. Node.js will pass you the data as an argument in the data event listener, and once that function executes buffer will empty again and if there is still some data in the underlying source buffer will automatically filled up with new data, and then the data event will be emitted again. NOTE: if you are listening for the data event and the readable event both will fire but the data event listener will first empty the buffer and THEN the readable event will fire so your read() will always return NULL.

var fs = require('fs'),
util = require('util');

var file = fs.createReadStream('./zeros.txt');
var dataSize = 0;

file.on('data', function() {
    // Once this listener will stop executing new data will be read
    // into the buffer and then the 'data' event will be emitted
    // again.
    console.log('data has been loaded and emptied!')
})

file.on('readable', function () {
    // Notice we want to try to read 1 byte from the buffer
    // but in the console we see that the read() method
    // resulted in NULL, which means that the buffer is empty.
    // That's of course because we enterd the flowing mode
    // by setting up the 'data' event. (In flowing mode)
    // after the execution of the 'data' event all data
    // from the buffer will be read, but the execution
    // of listeners will continue. After all the event listeners
    // attached to this stream will execute, Node.js will fill
    // the buffer automatically again.
    console.log('readable ' + file.read(1))
});

Last few results from the console:

data has been loaded and emptied!
readable null
data has been loaded and emptied!
readable null
data has been loaded and emptied!
readable null
data has been loaded and emptied!
readable null
data has been loaded and emptied!
readable null

Exhibitionist

Answer 2

My answer is based on the version of 0.12.4.

1: Every read(n) function which is extended from Stream.Readable will trigger the internal _read(n) function when the current internal buffer length is 0 or less than the value of highWaterMark property.

2: The readable event will only be triggered when the current internal buffer length is 0 or data read from the internal buffer is null or a null indicator has occurred.

Let's take your code as an example to see what has happened.

file.on('readable', function () {

A readable event handler register will trigger read(0) function to load the data from file to internal buffer. If you don't overwrite the value of highWaterMark, it will load 64 * 1024 = 65535 chunks at most. In your code, it loaded all the data from the file "zeros.txt". After that, it will trigger the readable event because the internal buffer length is 0 before the read(0) function invoked.

var data = file.read(10);

In the handler, you invoked read(10) function again. This also would trigger the loading process from file to buffer. However, no data would be loaded in this time. Therefore, null would be pushed to indicate that the reading process had been completed. The second readable event had been triggered. This is the reason you should see and only see two readable event.

If you read a file which size is larger than 65535 bytes (almost 66KB), you should only see one readable event triggered.

You should not write readable event handler like that, you should refer to the following:

var chunk;
while( null !== ( chunk = fs.read() ) ) {
    //handle the chunk
}

If you want to handle the chunk in your way to do some special things, please take care about the rules; otherwise, the program will stay on the "pause" status, no more data will be read and no more data will be retrieved.

Please see fs.ReadStream and stream.Readable.

Feeler