click commands in separate files

Question

I have a Python 3.6 script that uses latest click package. I have lots of commands so I would like to move some to separate modules. E.g.

main.py: root, with commands A and B as children
mylib1.py: commands C and D
mylib2.py: commands E and F

then in main I want to "import" C to F such that main appears to have all 6 commands, ie all 6 are children of root.

Brute force is lots of maintenance:

# mylib1.py
@click.command()
def cmd1():
   ...
...
@click.command()
def cmdN():
   ...

# main.py
import click
from mylib1 import cmd1, cmd2, ... cmdN

@click.group()
def main(): pass

@main.command()
def main_cmd1(): pass

main.add_command(cmd1)
...
main.add_command(cmdN)

A bit less maintenance (no imports to manage):

# mylib1.py
def add_commands(group):
    group.add_command(cmd1)
    ...
    group.add_command(cmdN)

# main.py
import click, mylib

@click.group()
def main(): pass

@main.command()
def main_cmd1(): pass

mylib1.add_commands(main)
main()

But the most maintainable seems to be to use click.CommandCollection, which allows me to make it appear as though commands in one group are actually in another:

# mylib1.py
import click

@click.group()
def commands():
    pass

@commands.command()
def cmd1():
   ...
...
@commands.command()
def cmdN():
   ...

# main.py
import click, mylib1

@click.group()
def main_group(): pass

@main_group.command()
def main_cmd1(): pass

main = click.CommandCollection([main_group, mylib1.commands])
main()

This seems to work well and so far no issues, but the documentation for CommandCollection says this is not "as recommended":

it can also be interesting to merge multiple together into one script. While this is generally not as recommended as it nests one below the other, the merging approach can be useful in some circumstances for a nicer shell experience.

Does anyone know what is meant by "not as recommended as it nests one below the other", what could be some possible gotchas to that approach?

Sherburn

Answer 1

Loading click commands from separate files

From the docs:

Click in three points:

• arbitrary nesting of commands

• automatic help page generation

• supports lazy loading of subcommands at runtime

I think the third point (which was considered important enough to put on that short list), points at the best way for what you are trying to do.

I tend to use click.CommandCollection only when I am not the owner (can't easily edit) some of the command code. I think for the use case you have here, the preferred solution is using click.MultiCommand. A description from the docs:

In addition to using click.group(), you can also build your own custom multi commands. This is useful when you want to support commands being loaded lazily from plugins.

There is a simple usage example for this in the docs, here on SO, and a more complex example on GitHub, which is described as:

complex is an example of building very complex cli applications that load subcommands dynamically from a plugin folder and other things.

If you need reasons to avoid CommandCollection, among other issues, there is this long standing bug on github titled: Adding group to CommandCollection loses group options.

Answering your question

As quoted in the question:

it can also be interesting to merge multiple together into one script. While this is generally not as recommended as it nests one below the other, the merging approach can be useful in some circumstances for a nicer shell experience.

Your question:

Does anyone know what is meant by "not as recommended as it nests one below the other", what could be some possible gotchas to that approach.

I think this rewrite might make the statement more clear.

While this is generally not as recommended as nesting one below the other, the merging approach can be ...

I believe that was what was trying to be said.

Riedel