Python: yield and yield assignment

Question

How does this code, involving assignment and the yield operator, work? The results are rather confounding.

def test1(x): 
    for i in x:
        _ = yield i 
        yield _
def test2(x): 
    for i in x:
        _ = yield i 

r1 = test1([1,2,3])
r2 = test2([1,2,3])
print list(r1)
print list(r2)

Output:

[1, None, 2, None, 3, None] 
[1, 2, 3]

Semiskilled

I'm not exactly sure why, but the one question most likely to appear in the "Related" list in any Python question is the yield question, even if the question in question has nothing to do with yield. This time, however, it is indeed related. – Penicillium

@TigerhawkT3: related questions are due to someone linking from one question to the other. So presumably there are a lot of comments pointing to that question. – Cameroun

Answer 1

The assignment syntax ("yield expression") allows you to treat the generator as a rudimentary coroutine.

First proposed in PEP 342 and documented here: https://docs.python.org/2/reference/expressions.html#yield-expressions

The client code that is working with the generator can communicate data back into the generator using its send() method. That data is accessible via the assignment syntax.

send() will also iterate - so it actually includes a next() call.

Using your example, this is what it would be like to use the couroutine functionality:

>>> def test1(x):
...     for i in x:
...         _ = yield i
...         yield _
...
>>> l = [1,2,3]
>>> gen_instance = test1(l)

>>> #First send has to be a None
>>> print gen_instance.send(None)
1
>>> print gen_instance.send("A")
A
>>> print gen_instance.send("B")
2
>>> print gen_instance.send("C")
C
>>> print gen_instance.send("D")
3
>>> print gen_instance.send("E")
E
>>> print gen_instance.send("F")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

Note that some of the sends are lost because of the second yield in each loop iteration that doesn't capture the sent data.

EDIT: Forgot to explain the Nones yielded in your example.

From https://docs.python.org/2/reference/expressions.html#generator.next:

When a generator function is resumed with a next() method, the current yield expression always evaluates to None.

next() is used when using the iteration syntax.

Rufe

Answer 2

_ = yield i 
yield _

First it yields the value referenced by i, e.g. 1. Then it yields the value returned by the yield operation, which is None. It does this on each iteration of the loop.

for i in x:
    _ = yield i

This simply yields the value referenced by i, e.g. 1, then proceeds to the next iteration of the loop, producing 2, then 3.

Unlike return, the yield keyword can be used in an expression:

x = return 0 # SyntaxError
x = yield 0 # perfectly fine

Now, when the interpreter sees a yield, it will generate the indicated value. However, when it does so, that operation returns the value None, just like mylist.append(0) or print('hello') will return the value None. When you assign that result to a reference like _, you're saving that None.

So, in the first snippet, you're yielding an object, then you save the "result" of that yield operation, which is None, and then you yield that None. In the second snippet, you yield an object, then you save the "result" of that yield operation, but you never yield that result, so None does not appear in the output.

Note that yield won't always return None - this is just what you sent to the generator with send(). Since that was nothing in this case, you get None. See this answer for more on send().

Penicillium

@JohnKugelman - Sure, I'll add some more detailed explanation. – Penicillium

@JohnKugelman - Information added. Please let me know if any of the wording is unclear or misleading. – Penicillium

The clearest explanation for me, thank you very much. – Protuberant

Answer 3

To expand on TigerhawkT3's answer, the reason that the yield operation is returning None in your code is because list(r1) isn't sending anything into the generator. Try this:

def test1(x):
    for i in x:
        _ = yield i
        yield _


r1 = test1([1, 2, 3])

for x in r1:
    print('   x', x)
    print('send', r1.send('hello!'))

Output:

   x 1
send hello!
   x 2
send hello!
   x 3
send hello!

Here's a somewhat manufactured example where sending values into a generator could be useful:

def changeable_count(start=0):
    current = start
    while True:
        changed_current = yield current
        if changed_current:
            current = changed_current
        else:
            current += 1

counter = changeable_count(10)

for x in range(20):
    print(next(counter), end=' ')

print()
print()

print('Sending 51, printing return value:', counter.send(51))
print()

for x in range(20):
    print(next(counter), end=' ')

print()
print()

print('Sending 42, NOT printing return value')
print()

counter.send(42)

for x in range(20):
    print(next(counter), end=' ')

print()

Output:

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 

Sending 51, printing return value: 51

52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 

Sending 42, NOT printing return value

43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62

Juna