event associated with fs.createWriteStream in node.js

Question

What event is triggered when EOF is reached while writing to a stream ? My code is as follows. and it is as per http://docs.nodejitsu.com/articles/advanced/streams/how-to-use-fs-create-write-stream

But surprisingly my 'end' event is never fired. When I checked http://nodejs.org/api/stream.html#stream_event_end, I see that writable stream does not have any event on 'end'

---

var x = a1.jpg;
var options1 = {'url': url_of_an_image, 'encoding': null};
var r = request(options1).pipe(fs.createWriteStream('/tmp/imageresize/'+x));

r.on('end', function(){
    console.log('file downloaded to ', '/tmp/imageresize/'+x);
}

How do I capture the EOF event ?

Blueweed

Answer 1

Updated 30 Oct 2013

Readable Steams emit close event when the underlying resource done writing.

r.on('close', function(){
  console.log('request finished downloading file');
});

But if you want to catch the moment when fs finished writing data to the disc, you need Writeable Stream finish event:

var w = fs.createWriteStream('/tmp/imageresize/'+x);

request(options1).pipe(w);

w.on('finish', function(){
  console.log('file downloaded to ', '/tmp/imageresize/'+x);
});

Charla

Does the close event mean that all of the content has been flushed to disk or could it still be in the OS write buffer? When is it safe to quit the process? – Leatherneck

I would bind the event to finish before calling request. It is a race condition that just happens to resolve in your favor, no? – Splitlevel

@LeoRomanovsky No, it's not a race condition, because I'm registering my event handler synchronously. Streams in node.js are asynchromous, so no events will be fired until the next tick of node.js event loop. – Charla

@LeonidBeschastny How is close event different from end event of Readable streams? – Fleenor

'end' event is emitted when Readable Stream is drained, i.e. when there is no more data to be consumed from it. 'close' event is emitted when the underlying resource is closed, i.e. when there is no more data to be written into it. It's all in the docs, by the way. – Charla

Bear in mind that close is called after an error is emitted: nodejs.org/api/stream.html#stream_event_error so finish is the one to go to know all was flushed to disk. – Bufordbug

so how does 'end' differ from 'finish' ? - it really is confusing . – Feune

the above link is for a readable stream - for a writable stream it should be nodejs.org/api/stream.html#class-streamwritable – Minus

Answer 2

This is really old but this still saved me a lot of time.

1. Solution from leonid-beschastny is correct. However, there is a very subtle difference in the complete sequence of the calls being made. The solutions are all from the experts and a newbie like me gets lost - due to simple differences like - how a stream is being created or the objects missed in the description.

2. Also, this is a generic solution to a lot of other scenarios. When you are piping to a writableStream - and by the time async function that contains this operation - returns. The streaming is not complete - so the probable image/document/pdf that is being piped to - is not yet created. It will be of size 0 KB.

3. Sharing here in simple steps.

   a. createPdfKitDocument with your definition as required.
   
   b. Create the writeStream the way you want as per your design.
   
   c. Pipe your prepared document to this stream.
   
   d. Add this finish event to your writeStream.
   
   e. Most important - without this it doesn't work.
   

Here

var pdfStream = printer.createPdfKitDocument(docDefinition); //a
let writerStream = fs.createWriteStream(fullfileName); //b
pdfStream.pipe(writerStream); //c
writerStream.on('finish', function () {
    //This could be your callback function.
    callback());
}) //d
pdfStream.end(); //e

4. I also referred to these links - they might help someone a. PDF download fails using PDFMake in NodeJS This gave me the clue to post.

5. Main issue is - createWriteStream is not finished [due to whatever reasn] - by the time the async function that contains it - returns.

6. Also leonid-beschastny is right in saying that race condition won't occur - I test my code by swapping the position of the pdfStream.pipe(writerStream);

and

writerStream.on('finish', function () {
    //This could be your callback function.
    callback());
})

Hope it helps few more people.

Chirr

The most important thing for me here was the 'finish' event. Thanks for your explanation and saving my time. – Evanescent