How to wait for all node streams to finish/end?

Question

I have a child Node process that runs on a CRON once every 24 hours. When the process begins, it reads some queued up data and shoves that data down some transform stream. This stream then acts as an inverse multiplexer and splits the stream into multiple streams which eventually resolve but they are all asynchronous.

I need to terminate this child process I create once all those streams have finished. My question is, how do you know when all of the streams have finished?

Attempts:

1. I have tried to use the EventEmitter's 'finish' event but that seems to be caught when the first inverse multiplexed stream finishes (thats a mouthful :)).

2. Promise based approach. So this approach works, but I figured there was an easier way to do such a thing. Basically, this results in creating a promise for each inverse multiplexed pipeline and when each of those pipelines finish we resolve that promise. Then when all promises have settled, an event is triggered and we catch that event somewhere else to terminate the process.

Driedup

I recommend looking into observables. bacon or rxjs ought to do it. – Bestow

I had similar problem solved with multiple child processes running and resolving using ramda-future and parallel-future modules - in very FP approach. It's similar to promise but since the tasks running/streams of child processes are more functional in their nature i preferred this, cleaner and leaner approach. – Araujo

Answer 1

I'd revisit your second option. Consider the following code to easily map your list to promises:

const promises = data.map((item) => functionThatReturnsPromise(item));

After you construct the promises, Promise.all is a way to execute multiple promises and return all results.

Promise.all(promises) .then(array_of_results => doStuff)

Gardia