Non-recursive method to iterate over Promise iterator

Question

I've developed a client library that exposes a method called iterator(). This method returns a Promise instance created using require('promise') library, which is completed with an iterator object.

This object contains a method called next() which returns a Promise which is completed with a complex object like this: {done: [true|false], key: _, value: _}

Although iterator() might pre-fetch some elements, next() needs to return a Promise in case it results in a remote call.

Now, say a user wants to iterate over all elements until the Promise returned by next() returns an object containing done: true.

I've managed to achieve this using the following recursive method (I originally found this solution in this answer):

var iterate = client.iterator();

iterateTeams.then(function(it) {

  function loop(promise, fn) {
    // Simple recursive loop over iterator's next() call
    return promise.then(fn).then(function (entry) {
      return !entry.done ? loop(it.next(), fn) : entry;
    });
  }

  return loop(it.next(), function (entry) {
    console.log('entry is: ' + entry);
    return entry;
  });

});

The question is, would it be possible, using require('promise') library, to build a non-recursive solution? The reason I'm interested in a non-recursive method is to avoid blowing up if the number of entries to iterate over is too big.

Cheers, Galder

Hesperides

It sounds like you're looking for a generator or observable. – Agosto

you can use another solution which hides the recursion for you but it still would be a recursive solution – Diaphragm

Galder, you probably want to read the section entitled "The Collection Kerfuffle" here – Fatuity

We're not there yet, but getting there :) See answer. – Capella

@Roamer-1888, thanks for the pointer but that section assumes you have an array or know how many elements you need to loop through and hence does not apply here. – Limbo

There is no reason why this would blow up. – Batiste

@Bergi, could you add that as s solution? – Limbo

Answer 1

The reason I'm interested in a non-recursive method is to avoid blowing up if the number of entries to iterate over is too big

Don't fear. Asynchronous "recursion" (sometimes dubbed pseudo-recursion) does not grow the call stack, it's much like tail recursion. You won't ever get a stackoverflow exception.

And if the promise library is implemented reasonably, this should not even grow the memory - see Building a promise chain recursively in javascript - memory considerations for details.

Batiste

Answer 2

Without new syntax or a library - generally no.

Well, if you're using babel, you can use ES2018 (:P) async iteration:

for await (const team of iterateTeams) {
   // do something with team
}

read more about it here

Otherwise, you can use generators with ES2016 async/await syntax:

for(var it = iterateTeams(); !done; ({done, value}) = await it.next()) {
    // work with value
}     

Or with available today ES2015 generator syntax and a pump through bluebird:

// inside a Promise.corutine  
for(var it = iterateTeams(); !done; ({done, value}) = yield it.next()) {
   // work with value
}     

Capella

Thanks a lot for those solutions but none of those work for me. I'm using ES2015 and require Node.js 0.10 (bluebird Promise.coroutine requires Node.js 0.12). – Limbo

@GalderZamarreño how are you using ES2015 if you're using NodeJS 0.10? – Capella

Also, the answer still stands, without recursion or a helper method that does recursion itself - it's impossible before ES2015 – Capella

Sorry, got confused, mean ES5 and Node.js 0.10. – Limbo

Then no, you have to use recursion or a helper function that would use recursion itself - effectively extracting your loop function to a helper. – Capella