for..of and the iterator state

Question

Consider this python code

it = iter([1, 2, 3, 4, 5])

for x in it:
    print x
    if x == 3:
        break

print '---'

for x in it:
    print x

it prints 1 2 3 --- 4 5, because the iterator it remembers its state across the loops. When I do seemingly the same thing in JS, all I get is 1 2 3 ---.

function* iter(a) {
    yield* a;
}

it = iter([1, 2, 3, 4, 5])

for (let x of it) {
    console.log(x)
    if (x === 3)
        break
}

console.log('---')

for (let x of it) {
    console.log(x)
}

What am I missing?

Sweetandsour

You have a generator and it is once and done. #23848613 – Muff

Answer 1

As pointed out in other answers, for..of closes the iterator in any case, so there's another wrapper necessary to preserve the state, e.g.

function iter(a) {
    let gen = function* () {
        yield* a;
    }();

    return {
        next() {
            return gen.next()
        },
        [Symbol.iterator]() {
            return this
        }
    }
}


it = iter([1, 2, 3, 4, 5]);

for (let x of it) {
    console.log(x);
    if (x === 3)
        break;
}

console.log('---');

for (let x of it) {
    console.log(x);
}

Sweetandsour

Answer 2

Generator objects in JS are not reusable unfortunately. Clearly stated on MDN

Generators should not be re-used, even if the for...of loop is terminated early, for example via the break keyword. Upon exiting a loop, the generator is closed and trying to iterate over it again does not yield any further results.

Delora

Yes, this (sadly) seems to be the answer. ECMA standard link ecma-international.org/ecma-262/7.0/… , item k. – Sweetandsour

Answer 3

As mentioned generators are a one off.

But it's easy to to simulate a re-usable iterator by wrapping the array inside a closure, and returning a new generator..

eg.

function resume_iter(src) {
  const it = src[Symbol.iterator]();
  return {
    iter: function* iter() {
      while(true) {
        const next = it.next();
        if (next.done) break;
        yield next.value;
      }
    }
  }
}

const it = resume_iter([1,2,3,4,5]);

for (let x of it.iter()) {
    console.log(x)
    if (x === 3)
        break
}

console.log('---')

for (let x of it.iter()) {
    console.log(x)
}



console.log("");
console.log("How about travesing the DOM");

const it2 = resume_iter(document.querySelectorAll("*"));

for (const x of it2.iter()) {
  console.log(x.tagName);
  //stop at first Script tag.
  if (x.tagName === "SCRIPT") break;
}

console.log("===");

for (const x of it2.iter()) {
  console.log(x.tagName);
}

Branen

Nice, but I'd like an iterator to be unaware of the underlying type, which is not necessary an array. – Sweetandsour

@Sweetandsour Oh, in that case how about wrapping an iterable inside a closure,.. Updated snippet to handle any iterable.. – Branen

@Sweetandsour Updated snippet to traverse DOM nodes, as that's not an array, basically stop at first SCRIPT tag, and then it resumes again.. – Branen

Answer 4

This behavior is expected according to the spec, but there is a simple solution. The for..of loop calls the return method after the loop ends:

Invoking this method notifies the Iterator object that the caller does not intend to make any more next method calls to the Iterator.

Solution

You can of course just replace that function with a custom one which does not close the actual iterator, right before utilizing it in a loop:

iter.return = value => ({ value, done: true });

Example:

function* iter(a) {
    yield* a;
}

it = iter([1, 2, 3, 4, 5])
it.return = () => ({})

for (let x of it) {
    console.log(x)
    if (x === 3)
        break
}

console.log('---')

for (let x of it) {
    console.log(x)
}

Disjoined

this is great! I didn't know about return – Sweetandsour

Answer 5

This has more to do with how the for..of operates than the reusability of the iterator. If you were to manually pull the next value of the iterator, you call it as many times as necessary and it would resume from the previous state.

Which makes something like this possible:

function* iter(a) {
  yield* a;
}

let values = [1, 2, 3, 4, 5];
let it = iter(values)

for (let i = 0, n = values.length; i < n; i++) {
  let x = it.next().value
  console.log(x)
  if (x === 3)
    break
}

console.log('---')

for (let x of it) {
  console.log(x)
}

And the same could be done for a while loop that isn't dependent on a values array:

function* iter(a) {
  yield* a;
}

let it = iter([1, 2, 3, 4, 5]),
  contin = true

while (contin && (x = it.next().value)) {
  console.log(x)
  if (x === 3)
    contin = false
}

console.log('---')

for (let x of it) {
  console.log(x)
}

The second example (while loop) deviates slightly as x is assigned during the condition evaluation. It assumes that all values of x are truthy so undefined can be used as a terminating condition. If that is not the case, it would need to be assigned in the loop block and a terminating condition would have to be set. Something like if(x===undefined)contin=false or checking if the iterator has reached the end of its inputs.

Denney

Good idea, I've posted a wrapper that does the same (manually pull values from an iterator thus preserving its state). – Sweetandsour

Answer 6

In addition to Andrey's answer, if you want to have the same functionality as in the Python script, since generators are not able to be re-used when the loop is exited, you can re-create the iterator before looping through each time and keep track of where the loop ends up being broken to exclude processing of already-processed results like so:

function* iter(a) {
  yield* a;
}

var broken = 0;

iterate();
console.log('---');
iterate();

function iterate() {
  var it = iter([1, 2, 3, 4, 5]);
  for (let x of it) {
    if (x <= broken)
      continue;
    console.log(x);
    if (x === 3) {
      broken = x;
      break;
    }
  }
}

Infinite

you are still looping twice. – Danieladaniele