ES6 generators mechanism - first value passed to next() goes where?

Question

When passing parameters to next() of ES6 generators, why is the first value ignored? More concretely, why does the output of this say x = 44 instead of x = 43:

function* foo() {
    let i = 0;
    var x = 1 + (yield "foo" + (++i));
    console.log(`x = ${x}`);
}

fooer = foo();

console.log(fooer.next(42));
console.log(fooer.next(43));

// output:
// { value: 'foo1', done: false }
// x = 44
// { value: undefined, done: true }

My mental model for the behavior of such a generator was something like:

1. return foo1 and pause at yield (and the next call which returns foo1 takes as argument 42)
2. pause until next call to next
3. on next yield proceed to the line with var x = 1 + 42 because this was the argument previously received
4. print x = 43
5. just return a {done: true} from the last next, ignoring its argument (43) and stop.

Now, obviously, this is not what's happening. So... what am I getting wrong here?

Cence

Calling foo() does not execute until the first yield, it only creates the generator object. The first next() call starts execution from the top of the function till the first yield - and returns the value yielded there. – Gulgee

Possible duplicate of In ES6, what happens to the arguments in the first call to an iterator's next method? – Gulgee

Answer 1

I ended up writing this kind of code to investigate the behavior more thoroughly (after re-re-...-reading the MDN docs on generators):

function* bar() {
    pp('in bar');
    console.log(`1. ${yield 100}`);
    console.log(`after 1`);
    console.log(`2. ${yield 200}`);
    console.log(`after 2`);
}
let barer = bar();
pp(`1. next:`, barer.next(1));
pp(`--- done with 1 next(1)\n`);
pp(`2. next:`, barer.next(2));
pp(`--- done with 2 next(2)\n`);
pp(`3. next:`, barer.next(3));
pp(`--- done with 3 next(3)\n`);

which outputs this:

in bar
1. next: { value: 100, done: false }
--- done with 1 next(1)

1. 2
after 1
2. next: { value: 200, done: false }
--- done with 2 next(2)

2. 3
after 2
3. next: { value: undefined, done: true }
--- done with 3 next(3)

So apparently the correct mental model would be like this:

• on first call to next, the generator function body is run up to the yield expression, the "argument" of yield (100 the first time) is returned as the value returned by next, and the generator body is paused before evaluating the value of the yield expression -- the "before" part is crucial

• only on the second call to next is the value of the first yield expression computed/replaced with the value of the argument given to next on this call (not with the one given in the previous one as I expected), and execution runs until the second yield, and next returns the value of the argument of this second yield -- here was my mistake: I assumed the value of the first yield expression is the argument of the first call to next, but it's actually the argument of the second call to next, or, another way to put it, it's the argument of the call to next during whose execution the value is actually computed

This probably made more sense to who invented this because the # of calls to next is one more times the number of yield statements (there's also the last one returning { value: undefined, done: true } to signal termination), so if the argument of the first call would not have been ignored, then the one of the last call would have had to be ignored. Also, while evaluating the body of next, the substitution would have started with the argument of its previous invocation. This would have been much more intuitive imho, but I assume it's about following the convention for generators in other languages too and consistency is the best thing in the end...

---

Off-topic but enlightening: Just tried to do the same exploration in Python, which apparently implements generators similar to Javascript, I immediately got a TypeError: can't send non-None value to a just-started generator when trying to pass an argument to the first call to next() (clear signal that my mental model was wrong!), and the iterator API also ends by throwing a StopIteration exception, so no "extra" next() needed just to check if the done is true (I imagine using this extra call for side effects that utilize the last next argument would only result in very hard to understand and debug code...). Much easier to "grok it" than in JS...

Cence

Yes, that's how it works. Taking the argument from the previous next call makes no sense, that's as if it was buffered. It would not allow to immediately "respond" to the passed value. – Gulgee

Given that generators implement passing "messages" back and forth (or in and out), one always needs to start somewhere. I guess it would also have been possible to have one more yield statement than next calls, by having the first yielded value go nowhere, but that's just ugly in the more common case. – Gulgee

@Gulgee ah, that makes sense, I didn't think at all in terms of "responsiveness"... guess that the "possible reaction to the value of an argument passed to next" is intuitive to happen during that actual call to next if I think of it this way! ...but throwing an error when passing an argument to first call to next, like Python I see does, would help newbs like me a loooot :) – Cence

Answer 2

I got this from Axel Rauschmayer's Exploring ES6, especially 22.4.1.1.

On receiving a .next(arg), the first action of a generator is to feed arg to yield. But on the first call to .next(), there is no yield to receive this, since it is only at the end of the execution.

Only on the second invocation x = 1 + 43 is executed and subsequently logged, and the generator ends.

Bluefield

That explanation is... interesting. I think I got it in the end by myself, see answer, neither your explanation nor the docs seem explicit enough to me: what does "feed arg to yield" or "no yield to receive this" even mean?! I think of code in terms of a formal model of evaluation and the "substitution model" (mitpress.mit.edu/sicp/full-text/sicp/book/node10.html) is the simplest for me to understand, not in terms of "feeding", "sending", "receiving" etc. ...these words don't make much sense to me unless I'm talking about multiple processes... – Cence

Answer 3

Also had a hard time wrapping my head around generators, especially when throwing in if-statements depended on yielded values. Nevertheless, the if-statement was actually what helped me getting it at last:

function* foo() {
  const firstYield = yield 1
  console.log('foo', firstYield)

  const secondYield = yield 3
  console.log('foo', secondYield)

  if (firstYield === 2) {
    yield 5
  }
}

const generator = foo()

console.log('next', generator.next( /* Input skipped */ ).value)
console.log('next', generator.next(2).value)
console.log('next', generator.next(4).value)

/* 
  Outputs:

  next 1
  foo 2
  next 3
  foo 4
  next 5    
*/

Bouncer

Answer 4

Everything immediately became clear once I made this realization.

Here's your typical generator:

function* f() {
  let a = yield 1;
  // a === 200
  let b = yield 2;
  // b === 300
}

let gen = f();
gen.next(100) // === { value: 1, done: false }
gen.next(200) // === { value: 2, done: false }
gen.next(300) // === { value: undefined, done: true }

But here's what actually happens. The only way to make generator execute anything is to call next() on it. Therefore there needs to be a way for a generator to execute code that comes before the first yield.

function* f() {
  // All generators implicitly start with that line
  // v--------<---< 100
       = yield
  // ^-------- your first next call jumps right here

  let a = yield 1;
  // a === 200
  let b = yield 2;
  // b === 300
}

Cooking