Return and yield in the same function

Question

What exactly happens, when yield and return are used in the same function in Python, like this?

def find_all(a_str, sub):
    start = 0
    while True:
        start = a_str.find(sub, start)
        if start == -1: return
        yield start
        start += len(sub) # use start += 1 to find overlapping matches

Is it still a generator?

Coorg

Answer 1

Yes, it' still a generator. The return is (almost) equivalent to raising StopIteration.

PEP 255 spells it out:

Specification: Return

A generator function can also contain return statements of the form:

"return"

Note that an expression_list is not allowed on return statements in the body of a generator (although, of course, they may appear in the bodies of non-generator functions nested within the generator).

When a return statement is encountered, control proceeds as in any function return, executing the appropriate finally clauses (if any exist). Then a StopIteration exception is raised, signalling that the iterator is exhausted. A StopIteration exception is also raised if control flows off the end of the generator without an explict return.

Note that return means "I'm done, and have nothing interesting to return", for both generator functions and non-generator functions.

Note that return isn't always equivalent to raising StopIteration: the difference lies in how enclosing try/except constructs are treated. For example,

>>> def f1():
...     try:
...         return
...     except:
...        yield 1
>>> print list(f1())
[]

because, as in any function, return simply exits, but

>>> def f2():
...     try:
...         raise StopIteration
...     except:
...         yield 42
>>> print list(f2())
[42]

because StopIteration is captured by a bare "except", as is any exception.

Deflation

Do you know what would happen if the return had an argument? – Southworth

@Zack In Python 2.x, it'd be a SyntaxError: SyntaxError: 'return' with argument inside generator. It's allowed in Python 3.x, but is primarily meant to be used with coroutines - you make asynchronous calls to other coroutines using yield coroutine() (or yield from coroutine(), depending on the asynchronous framework you're using), and return whatever you want to return from the coroutine using return value. In Python 2.x, you need to use a trick like raise Return(value) to return values from coroutines. – Bosomy

Answer 2

Yes, it is still a generator. An empty return or return None can be used to end a generator function. It is equivalent to raising a StopIteration(see @NPE's answer for details).

Note that a return with non-None arguments is a SyntaxError in Python versions prior to 3.3.

As pointed out by @BrenBarn in comments starting from Python 3.3 the return value is now passed to StopIteration.

From PEP 380:

In a generator, the statement

return value

is semantically equivalent to

raise StopIteration(value)

Whipstock

Do you know what would happen if the return had an argument (other than None)? – Southworth

In Python 3.3 and up, you can use return with an argument to pass the argument to the StopIteration that is raised. See this question. – Algae

@AshwiniChaudhary The coroutine implementation in the new asyncio module is built on that feature (along with the yield from keyword). – Bosomy

@Bosomy All of it started with PEP 342 right? – Whipstock

@AshwiniChaudhary That enabled basic coroutines in Python - the ability to send values/exceptions into generators, and receive them via value = yield, etc. The introduction of yield from and the ability to return values from generators came with PEP 380, both of which are leveraged leveraged by asyncio. You can still have a robust coroutine implementation with just the features provided by PEP 343, it's just a little less clean to write them. – Bosomy

It's probably worth explicitly noting this will just silently ignore the returned value in a for-loop, for example, which can be quite counter-intuitive. – Bobbie

@AshwiniChaudhary saying that raise StopIteration() is equivalent to return is misleading. If you raise the exception instead of using return, the exception will NOT be catched by the generator function and will be thrown as any other exception. – Tetanus

Answer 3

There is a way to accomplish having a yield and return method in a function that allows you to return a value or generator.

It probably is not as clean as you would want but it does do what you expect.

Here's an example:

def six(how_many=None):
    if how_many is None or how_many < 1:
        return None  # returns value

    if how_many == 1:
        return 6  # returns value

    def iter_func():
        for count in range(how_many):
            yield 6
    return iter_func()  # returns generator

Urien

not clear "that it does do what you expect". could you provide an example of how your approach could be put to good use? – Advised

"that it does do what you expect" as in the subject of the question "Return and yield in the same function". Personally, I have switched to Haskell and this could be used/managed well with an Algebraic Data Type but with Python it is hard enough to manage your types and this does not fit well in the Python type declarations. So, you if you are asking the question of how this could be put to good use please do not use it. Otherwise, this allows you to return no, a single, or multiple values. This could be used to effectively traverse a tree. – Urien

Answer 4

Note: you don't get StopIteration exception with the example below.

def odd(max):
    n = 0
    while n < max:
        yield n
        n = n + 1
    return 'done'


for x in odd(3):
    print(x)

The for loop catches it. That's its signal to stop

But you can catch it in this way:

g = odd(3)

while True:
    try:
        x = next(g)
        print(x)
    except StopIteration as e:
        print("g return value:", e.value)
        break

Benildis