itertools.tee on a coroutine?

Question

I have a tree structure of objects. I need to iterate over all the items ("values") in the leaves. For this I'm currently using generator methods as illustrated below:

class Node(object):
    def __init__(self):
        self.items = [Leaf(1), Leaf(2), Leaf(3)]

    def values(self):
        for item in self.items:
            for value in item.values():
                yield value

class Leaf(object):
    def __init__(self, value):
        self.value = value

    def values(self):
        for i in range(2):
            yield self.value

n = Node()
for value in n.values():
    print(value)

This prints:

1
1
2
2
3
3

Now, the values returned by a Leaf will depend on an external parameter. I was thinking of employing coroutines to be able to pass this parameter down to the leaf nodes:

import itertools

class Node2(object):
    def __init__(self):
        self.items = [Leaf2(1), Leaf2(2), Leaf2(3)]

    def values(self):
        parameter = yield
        for item in self.items:
            item_values = item.values()
            next(item_values)    # advance to first yield
            try:
                while True:
                    parameter = (yield item_values.send(parameter))
            except StopIteration:
                pass

class Leaf2(object):
    def __init__(self, value):
        self.value = value

    def values(self):
        parameter = yield
        try:
            for i in range(2):
                parameter = (yield '{}{}'.format(self.value, parameter))
        except StopIteration:
            pass

n2 = Node2()
values2 = n2.values()
next(values2)    # advance to first yield

try:
    for i in itertools.count(ord('A')):
        print(values2.send(chr(i)))
except StopIteration:
    pass

This code is far from pretty, but it works. It prints:

1A
1B
2C
2D
3E
3F

There's a problem with this solution though. I was using itertools.tee (and chain) extensively to easily save the state of the iterator in case I needed to backtrack. I was hoping these would work on coroutines as well, but alas, no such luck.

Some alternative solutions I'm considering at the moment:

• have the generators yield functions (closures) that accept the external parameter
• write custom classes to emulate coroutines with capabilities for saving state

The first option seems the most attractive. But perhaps there are better options?

---

Some context: I'm using this construct in RinohType where the tree is formed by MixedStyledText (node) and SingleStyledText (leaf) objects. The spans() methods yield SingleStyledText instances. The latter can be dependent on external parameters. For example, the page number they are being rendered to. These are currently treated as a special case.

Wallack

What version of Python are you using? – Karelia

@GamesBrainiac I'm developing on Python 3.4, but will probably backport to 2.7 eventually (single codebase for 2.7 and 3.x). Any particular reason you ask? – Wallack

Sounds like a place where the visitor pattern could be applied - have you considered that? – Evacuation

Answer 1

To the extent possible, I like to make functions return simple data structures. In this case,

• I'd make each node yield a simple dictionary (or, in RinohType's case, a SingleStyledTextConfig object or namedtuple). This implementation will play nicely with itertools, as before, since it just transforms data into other data.
• I'd transform this collection of objects to account for external parameters (like the page number).
• Finally, I'd use the collection of configuration data to actually create the output (in RinohType's case, the SingleStyledText object).

At a higher level: your proposed solution (as I understand your simplified version) is trying to do too much stuff at the same time. It's trying to configure object creation, tweak that configuration, and create objects based on that configuration all in one step. Your implementation will be simpler, play nicer with itertools, and be easier to test if you separate these concerns.

For a more detailed treatment of this kind of thinking, see Gary Bernhardt's Boundaries talk and Brandon Rhodes' talk on the Clean Architecture in Python (and of course the resources they mention in the talks).

Imperfect

Answer 2

This is a start on the second option

from types import GeneratorType

def gen_wrapper(func):
    def _inner(*args):
        try:
            if args:
                if isinstance(args[0], GeneratorType):
                    func.gen = getattr(func, 'gen', args[0])

                func.recall = next(func.gen)

            try:
                return func.recall
            except AttributeError:
                func.recall = next(func.gen)
                return func.recall 

        except StopIteration:
            pass
    return _inner

@gen_wrapper
def Gen_recall(*args):
    pass                    

Damick

Answer 3

I'm not sure if I understand the question correctly. Is it needed for Leaf2 to do the calculations? If not, then you can do:

import itertools

class Node2(object):
    def __init__(self):
        self.items = [Leaf2(1), Leaf2(2), Leaf2(3)]

    def values(self):
        for item in self.items:
            for value in item.values():
                yield value

class Leaf2(object):
    def __init__(self, value):
        self.value = value

    def values(self):
        for i in range(2):
            yield self.value

def process(i, parameter):
    return '{}{}'.format(i, parameter)

n = Node2()
for value, parameter in zip(n.values(), itertools.count(ord('A'))):
    print(process(value, chr(parameter)))

If it's really important that Leaf2 does the processing, you could do

class Leaf2:
    def values(self):
        for i in range(2):
            yield self

def process(self, parameter):
    pass

So that in the main you can do

n = Node2()
for node, parameter in zip(n.values(), itertools.count(ord('A'))):
    print(node.process(chr(parameter)))

Orten