Getting the value of a return after a yield in python [duplicate]

Question

I would like to know how to get the return value of a function after all the execution of yield in a function like this:

def gen_test():
    l = []
    for i in range(6):
        l.append(i)
        yield i
    # i want to know this value after all iteration of yield
    return l

Whitelivered

Could you make it into a class method and then collect that value via a separate getter method in the same class (if you store it inside self)? – Painty

A return and a yield in the same function returns an error in py2 - it's a;llowed in py3 but for specific use-cases with co-routines. See this question for a good summary of the mechanisms behind this: #26596395 – Juvenal

If the return part works, then just return l and i like: return l, i – Outflow

@Juvenal I don't interpret this as saying that the OP specifically wants to use a return statement, but that they want to get the value of l somehow and are asking how. – Painty

Probably you shouldn't be using a generator in the first place? Since this generator keeps references on each item yielded, it's not more efficient than using a list. – Dvinsk

Answer 1

The short version is that it is not allowed. Passing values with the return statement in generators causes an error in python prior to version 3.3. For these versions of Python, return can only be used without an expression list and is equivalent to raise StopIteration.

For later versions of Python, the returned values can be extracted through the value-attribute of the exception.

You can find more information about this here: Return and yield in the same function

Forthright

This is false. It totally is allowed. – Ssw

How is it allowed? The poster is asking if it is possible to pass expressions with the return statement. – Forthright

See my answer. It is allowed. Normal generator functions return None because it's hard to get the return value because for obscures the underlying exception. That doesn't mean it's not used. – Ssw

I stand corrected. Updated the answer. – Forthright

Thanks. Vote flipped. Nice mention of 3.3- – Ssw

Answer 2

The return value of your iterator function is used as the argument to the StopIteration that it raises when it terminates:

>>> it = gen_test()
>>> next(it)
0
>>> next(it)
1
>>> next(it)
2
>>> next(it)
3
>>> next(it)
4
>>> next(it)
5
>>> next(it)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration: [0, 1, 2, 3, 4, 5]

Only the first StopIteration gets the specified value. Further attempts to iterate an empty generator will raise an empty exception:

>>> next(it)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

You can extract the return value from a normal run of the iterator by avoiding for loops and trapping the exception yourself:

it = gen_test()
while True:
    try:
        x = next(it)
        # Do the stuff you would normally do in a for loop here
    except StopIteration as e:
        ret = e.value
        break
print(ret)

All this is an awkward approach given that you have to decide between not using for loops and not being able to return your accumulated data. Assuming that you're not OK with trying to accumulate the data externally (e.g. just doing list(gen_test())), you can make your generator's state publicly accessible by using a class:

class gen_test:
    def __init__(self):
        self.l = []
        self.it = iter(range(6))

    def __iter__(self):
        return self

    def __next__(self):
        i = next(self.it)
        self.l.append(i)
        return i

Now you can do something like

>>> it = gen_test()
>>> for x in it:
...     print(x)
0
1
2
3
4
5
>>> print(it.l)
[0, 1, 2, 3, 4, 5]

Ssw

Do you mean value rather than arg? – Painty

@alani. I meant args[0]. Fixed now – Ssw

e.value is better (args[0] can raise IndexError for bare return). Also worth mentioning: you only have one chance to get this, subsequent StopIteration won't have it. So if the generator is consumed with a for loop, you'll have to chain another generator or do with a decorator or something. I am not the downvoter. – Dvinsk

@wim. Well seen. Will update momentarily. – Ssw

@Dvinsk Fixed both. Answer is technically correct. Now I'll add a better way to do this. – Ssw

I think this is a complete and precise answer, and seems awkward because the Python implementation is, itself awkward. – Provence

Seems the popular solution is to wrap it and use the result of yield from. – Sepoy

@KellyBundy. Very nice find. This is what happens when I don't bother to look at the spec. – Ssw

Answer 3

The presence of yield in your function turns it into something different than a regular function - python calls it a generator. You can not call a generator and expect it to preform like a function. Except perhaps in a degenerate or simplistic case. To retrieve state information ( like the whole list l ) in your question at the end of the generator life, you need to store in in a variable or return it every time in the yield statement.

To return it every time the generator is called do this:

def gen_test():
    l = []
    for i in range(6):
        l.append(i)
        yield i, l
    
for y,l in gen_test():
        print (y,l)
    

Otherwise you could declare your generator as part of a class and store intermediate results in a class variable. Then retrieve that variable value using a different call later.

Provence

The return value of a generator is meaningful, though rarely used. It gets assigned to the value of the StopIteration that is raised at the end. – Ssw

Answer 4

You can't easily use return in a generator function (a function that contains yield) to return an expression, because calling the function returns a generator. return in a generator is the same as raising StopIteration and if you return a value, that gets stuffed into the ecxeption. Since most ways of iterating in Python don't give you access to the StopIteration, this value is difficult to get at, unless you eschew for and most other ways to iterate (e.g. list()).

You could raise an exception with your desired return value:

class ReturnValue(Exception):
    pass

def gen_test():
    l = []
    for i in range(6):
        l.append(i)
        yield i
    raise ReturnValue(l)

try:
    for x in gen_test():
        print(x)
except ReturnValue as e:
    print(e.args[0])  # or
    returnvalue = e.args[0]
    # note: variable e goes away after exception handler

This seems like a hack, though. A class might be better:

class GenTest:
    value = None
    def __iter__(self):
        l = []
        for i in range(6):
            l.append(i)
            yield i
        self.value = l

for x in (it := GenTest()):
    print(x)

print(it.value)

Oilskin

The returned value initializes StopIteration. See my answer – Ssw

@MadPhysicist That's nice, but having to eschew for loops and other iteration methods is like cutting off your foot. I prefer the class solution, frankly. – Oilskin

@MadPhysicist True, though I've given him an up-vote for including a solution based on a class (per my original comment on the question), because it has the advantage that you can iterate using ordinary methods - in this example a for loop but there are many other ways that you might want to iterate e.g. list(it) and having to catch StopIteration manually may be inconvenient. – Painty

Given our understanding, the wording of the first sentence or two needs to be changed. I agree that giving up for loops is pointless, but the fact remains that return and yield can appear in the same def meaningfully. – Ssw

Also, your class implementation is much nicer than mine. – Ssw

@MadPhysicist kindall's version is the obvious equivalent of the OP's original code (which is what I had in mind with my original comment). But it is also helpful to have your class implementation using __next__ so that people can see both. – Painty