Python generator function/object naming convention

Question

I have a few logical processes implemented in the same class. A class instance get a generator for each process, and run() advances said generators. In my case generators don't end.

How would you call foo_function and foo_object in the code below

class C(threading.Thread):
    def foo_function(self):
        """ generator *function*,
            logical process foo """
        while True:
            # some state checks
            if self.some_attr:
                # side-effects here
                pass
            yield

    def __init__(self):
        # generator *object*
        # i.e. process instance
        self.foo_object = self.foo_function()  # <- here

    def run(self):
        while True:
            next(self.foo_object)
            next(self.another_object)
            if xxx:
                next(self.yet_another_object)

Typical processes are discovery, authentication, watchdog, etc.

How can I name function that defines the generator and the attribute that contains generator object in a sensible way?

Finally just for the kicks, same name name would be insane, right?

class C:
    def foo(self):
        yield 1; yield 2
    def __init__(self):
        self.foo = self.foo()

c = C()            
type(C.foo) is function
type(c.foo) is generator

Crowning

Lads, how about you leave a comment what you consider opinion-based? After all it deals with martinfowler.com/bliki/TwoHardThings.html :) – Crowning

As written, yes this is very confusing. :) Do you have some concrete specifics that might help to clarify this very abstract example? Details: foo() is run purely for side-effect? self.maybe_foo = foo_if_some_attr() – Dynamoelectric

you really lost me on this line: self.name_me_process_generator = self.name_me_process_function() – Dynamoelectric

Good point, I'll attempt to re-edit. – Crowning

Answer 1

You could make the members containing generators be created automatically from the function's name, using some kind of covention designed by your self. For instance, in my convention all generators will be contained by a member called: <function_name>_gen.

I would call the function name as its responsability: discovery, authentication and watchdog are good names. So you only need a way to automatically set: self.discovery_gen, self.authentication_gen and self.watchdog_gen.

Code sample:

class C:

    def discovery(self):
        yield 1; yield 2

    # Register all the functions for wich you want to have a generator
    # object.
    REGISTERED_GEN_FUNCTIONS = [discovery]

    def __init__(self):

        for func in self.REGISTERED_GEN_FUNCTIONS:
            name = func.__name__
            # All generators objects will be called <function_name>_gen.
            setattr(self, "{0}_gen".format(name), getattr(self, name)())

a = C()
for i in a.discovery_gen:
    print(i)

Ouput

>>> 1
>>> 2 

Bumboat

Cool idea to instantiate all generators in one go. – Crowning

This was a close call. !Raydel actually provided a naming convention, and @parchment pointed to using anonymous objects to avoid naming convention at all. I didn't like this naming convention, and avoidance is an answer to a different question. Thus, the prize is shared, accepted answer to one respondent and bounty to the other. – Crowning

Making method aliases like that at runtime looks like a very bad idea as IDEs will not be able to find the method's definition and the linter will growl. Instead just explicitly name them xxx_gen(). – Flibbertigibbet

@HenriTel, Well, the IDE not being able to find the name of the dynamically created generator instances (not method aliases), is not a problem for all of us. I would call a problem something that could cause my code to malfunction. Having said that, we can find solutions in many languages (generally when dealing with metaprogramming) that still are very good ideas even if the IDE is not able to recognize or foresee all the components. – Bumboat

@RaydelMiranda It's not just about the IDE. For an interpreted language like python, linting is often a very important CI step because it can catch errors that are otherwise found at runtime, often in prod. – Flibbertigibbet

@HenriTel, you have unit testing to cover that. Linter should not be used for testing on functionality. Imagine you're using dependency injection (passing types, functions, or even other modules as parameters) the linter has little to help in those cases, however, unit testing might save your day. For CI, I would recommend use the linter for an extra check, but the heavy lifting for ensuring everything is OK should be done by unit tests. – Bumboat

Answer 2

If the generators are only accessed inside the run method you can do something like this:

class C:
    def a_process(self):
        while True: yield

    def another_process(self):
        while True: yield

    def run(self):
        # Use itertools.izip in python 2
        for _ in zip(a_process(), another_process()):
            pass

Here, zip and the for loop will create and advance the generators automatically. This way, you don't need to keep track of them.

If you need access to the generators outside the run method, you can create an ordered dictionary of generators (in case the generators need to be advanced in a defined order):

from collections import OrderedDict

class C:
    # Processes here

    def __init__(self):
        self.generators = OrderedDict()
        for gn in ('a_process', 'another_process'):
            self.generators[gn] = getattr(self, gn)()

    def run(self):
        while True:
            for g in self.generators.values():
                next(g)

Friedman

Interesting take on the problem. A funny side-note is that I was able to refactor my code to use iterools.izip; This is in a way similar to scoping (keep function ref's in a class and instance ref's as local variables). However I am still interested in the answer to the original problem. How to name things to keep generator function and instance unambiguous. – Crowning

This was a close call. @Raydel actually provided a naming convention, and !parchment pointed to using anonymous objects to avoid naming convention at all. I didn't like this naming convention, and avoidance is an answer to a different question. Thus, the prize is shared, accepted answer to one respondent and bounty to the other. – Crowning

Answer 3

Addressing only the headline "python generator function naming convention".

Personally, I like to name generator functions like "yield_items". The code reads naturally and the keyword yield indicates that it outputs a generator object.

def yield_squares(iter_max=10):
    for i in range(iter_max):
        yield i**2

squares = yield_squares()
for square in squares:
    print(square)

Undercarriage

Interesting idea! Explicit yield_smth feels a bit Python2 though, doesn't it? I mean Python3 returns iterators for map, filter, reversed, thus perhaps it's just standard now? – Crowning