What happens when you invoke a function that contains yield?

Question

I read here the following example:

>>> def double_inputs():
...     while True:      # Line 1
...         x = yield    # Line 2
...         yield x * 2  # Line 3
...
>>> gen = double_inputs()
>>> next(gen)       # Run up to the first yield
>>> gen.send(10)    # goes into 'x' variable

If I understand the above correctly, it seems to imply that Python actually waits until next(gen) to "run up to" to Line 2 in the body of the function. Put another way, the interpreter would not start executing the body of the function until we call next.

1. Is that actually correct?
2. To my knowledge, Python does not do AOT compilation, and it doesn't "look ahead" much except for parsing the code and making sure it's valid Python. Is this correct?
3. If the above are true, how would Python know when I invoke double_inputs() that it needs to wait until I call next(gen) before it even enters the loop while True?

Alidaalidade

Answer 1

Correct. Calling double_inputs never executes any of the code; it simply returns a generator object. The presence of the yield expression in the body, discovered when the def statement is parsed, changes the semantics of the def statement to create a generator object rather than a function object.

Charcot

Answer 2

The function contains yield is a generator.

When you call gen = double_inputs(), you get a generator instance as the result. You need to consume this generator by calling next.

So for your first question, it is true. It runs lines 1, 2, 3 when you first call next.

For your second question, I don't exactly get your point. When you define the function, Python knows what you are defining, it doesn't need to look ahead when running it.

For your third question, the key is yield key word.

Carlyncarlynn

Answer 3

Generator-function is de iure a function, but de facto it is an iterator, i.e. a class (with implemented __next__(), __iter()__, and some other methods.)

          In other words, it is a class disguised as a function.

It means, that “calling” this function is in reality making an instance of this class, and explains, why the “called function” does initially nothing. This is the answer to your 3^rd question.

---

The answer to your 1^st question is surprisingly no.

Instances always wait for calling its methods, and the __next__() method (indirectly launched by calling the next() build-in function) is not the only method of generators. Other method is the .send(), and you may use gen.send(None) instead of your next(gen).

---

The answer to your 2^nd question is no. Python interpreter by no mean "look ahead" and there are no exceptions, including your

... except for parsing the code and making sure it's valid Python.

Or the answer to this question is yes, if you mean “parsing only up to the next command”. ;-)

Joeljoela

You may see a more complex picture in my other illustrated answer. – Joeljoela