How to sequentially write a stream in Node in a loop with writeable Streams?
Asked Answered
E

2

6

I want to append lines to a file, which is a writeable stream, in Node, in a specific order, through a loop.

Here is a simplified version of what I am talking about:

var fs = require('fs');
var outfile = fs.createWriteStream('outfile.txt');

for (var i = 0; i < 200; i++) {
    outfile.write('i', function (err, len) {
        console.log('written');
    });
}

I need the file to be written sequentially meaning that it would write, 12345...etc in order. My understanding is that outfile.write is async, meaning that it is possible for the file to be written out of order - is this correct?

I see from the node docs

Note that it is unsafe to use fs.write multiple times on the same file without waiting for the callback. For this scenario, fs.createWriteStream is strongly recommended.

So I believe that using a stream is best, however, I am not sure hot to guarantee that the data chunk was written succesfully before moving on to the next iteration in the loop. How should I go about doing that? Thanks.

Ellipticity answered 18/11, 2015 at 3:27 Comment(0)
Q
0

I think with both the current streams and the streams you had in 2015 the write was never async parallel. Write streams have a buffer in memory and you write to that buffer not to the hard disk directly. The buffer will keep your sequence in the right order.

Quinby answered 2/2, 2021 at 23:32 Comment(0)
Q
-3

You can avoid using fs.createWriteStream by using fs.writeSync which doesn't have a callback problem like fs.write

Quieten answered 18/11, 2015 at 4:26 Comment(3)
I am not a fan of this because of the synchronous and blocking nature of the code.Ellipticity
In your scenario, blocking is what is needed by definition of the problem.Celia
@RūdolfsVikmanis, the Writable object has been designed for writing sequential data. That's why it is called stream, so using writeSync() when you can take advantage of streaming functionality is foolish.Cannibalize

© 2022 - 2024 — McMap. All rights reserved.