Adding common parameters to groups with Click

Question

I am trying to use the Python library Click, but struggle to get an example working. I defined two groups, one of which (group2) is meant to handle common parameters for this group of commands. What I want to achieve is that those common parameters get processed by the group function (group2) and assigned to the context variable, so they can be used by the actual commands.

A use case would be a number of commands that require username and password, while some others don't (not even optionally).

This is the code

import click


@click.group()
@click.pass_context
def group1(ctx):
    pass


@click.group()
@click.option('--optparam', default=None, type=str)
@click.option('--optparam2', default=None, type=str)
@click.pass_context
def group2(ctx, optparam):
    print 'in group2', optparam
    ctx['foo'] = create_foo_by_processing_params(optparam, optparam2)


@group2.command()
@click.pass_context
def command2a(ctx):
    print 'command2a', ctx['foo']


@group2.command()
@click.option('--another-param', default=None, type=str)
@click.pass_context
def command2b(ctx, another_param):
    print 'command2b', ctx['foo'], another_param

# many more more commands here...
# @group2.command()
# def command2x():
# ...


@group1.command()
@click.argument('argument1')
@click.option('--option1')
def command1(argument1, option1):
    print 'In command2', argument1, option1

cli = click.CommandCollection(sources=[group1, group2])


if __name__ == '__main__':
    cli(obj={})

And this is the result when using command2:

$ python cli-test.py command2 --optparam=123
> Error: no such option: --optparam`

What's wrong with this example. I tried to follow the docs closely, but opt-param doesn't seem to be recognised.

Whim

It is not clear what command line you are trying to achieve. Do you want the --optparam to modify the group or the command? In the case of the above code you are applying the option to the group, but in all use cases you are using it in the command. Why not simply apply the option to the command directly? – Karelia

You are right. It's a bit unclear. I have many group2 commands that have mainly common options. Those options should be evaluated and processed and based on that the context variable should be updated (ideally in the group and passed as context to the commands). I'll update the example soon. – Whim

@StephenRauch: See the updated example and please let me know if anything is still unclear. – Whim

OK, so if I understand correctly, you want (on the command line) to apply the options to the command, but you want to decorate the options on the group, and do some processing in the group function, and then save that into the ctx, so that the command can use the info. This is achievable, but what do you see as the advantage of doing this on the group instead of on each command? – Karelia

Yes, that's the intent. The advantage is just (IMHO) cleaner code (i.e. processing (and defining) arguments once instead of remembering to do it at every command of the group). I can add commands to the group without worrying about any processing required. – Whim

Answer 1

The basic issue with the desired scheme is that click.CommandCollection does not call the group function. It skips directly to the command. In addition it is desired to apply options to the group via decorator, but have the options parsed by the command. That is:

> my_prog my_command --group-option

instead of:

> my_prog --group-option my_command

How?

This click.Group derived class hooks the command invocation for the commands to intercept the group parameters, and pass them to the group command.

1. In Group.add_command, add the params to the command
2. In Group.add_command, override command.invoke
3. In overridden command.invoke, take the special args inserted from the group and put them into ctx.obj and remove them from params
4. In overridden command.invoke, invoke the group command, and then the command itself

Code:

import click

class GroupWithCommandOptions(click.Group):
    """ Allow application of options to group with multi command """

    def add_command(self, cmd, name=None):
        click.Group.add_command(self, cmd, name=name)

        # add the group parameters to the command
        for param in self.params:
            cmd.params.append(param)

        # hook the commands invoke with our own
        cmd.invoke = self.build_command_invoke(cmd.invoke)
        self.invoke_without_command = True

    def build_command_invoke(self, original_invoke):

        def command_invoke(ctx):
            """ insert invocation of group function """

            # separate the group parameters
            ctx.obj = dict(_params=dict())
            for param in self.params:
                name = param.name
                ctx.obj['_params'][name] = ctx.params[name]
                del ctx.params[name]

            # call the group function with its parameters
            params = ctx.params
            ctx.params = ctx.obj['_params']
            self.invoke(ctx)
            ctx.params = params

            # now call the original invoke (the command)
            original_invoke(ctx)

        return command_invoke

Test Code:

@click.group()
@click.pass_context
def group1(ctx):
    pass

@group1.command()
@click.argument('argument1')
@click.option('--option1')
def command1(argument1, option1):
    click.echo('In command2 %s %s' % (argument1, option1))


@click.group(cls=GroupWithCommandOptions)
@click.option('--optparam', default=None, type=str)
@click.option('--optparam2', default=None, type=str)
@click.pass_context
def group2(ctx, optparam, optparam2):
    # create_foo_by_processing_params(optparam, optparam2)
    ctx.obj['foo'] = 'from group2 %s %s' % (optparam, optparam2)

@group2.command()
@click.pass_context
def command2a(ctx):
    click.echo('command2a foo:%s' % ctx.obj['foo'])

@group2.command()
@click.option('--another-param', default=None, type=str)
@click.pass_context
def command2b(ctx, another_param):
    click.echo('command2b %s %s' % (ctx['foo'], another_param))

cli = click.CommandCollection(sources=[group1, group2])

if __name__ == '__main__':
    cli('command2a --optparam OP'.split())

Results:

command2a foo:from group2 OP None

Karelia

This is a dirty trick. 1) ctx.invoked_subcommand wont' be initialised 2) invoke_without_command won't work - group function won't be called without param, instead you will get help 3) Most probably will allow to run chain fo two commands in non-chain mode – Pinckney

This is not quite what I am looking for (sorry to necro), I just want a syntax for common arguments and/or parameters for commands, e.g. @click.group(common_args=['argname1','argname2'],common_params=[]) This doesn't seem to exist... @Pinckney perhaps you know something more? – Rebato

Answer 2

This isn't the answer I am looking for, but a step towards it. Essentially a new kind of group is introduced (GroupExt) and the option added to the group is now being added to the command.

$ python cli-test.py command2 --optparam=12
cli
command2 12


import click


class GroupExt(click.Group):
    def add_command(self, cmd, name=None):
        click.Group.add_command(self, cmd, name=name)
        for param in self.params:
            cmd.params.append(param)


@click.group()
def group1():
    pass


@group1.command()
@click.argument('argument1')
@click.option('--option1')
def command1(argument1, option1):
    print 'In command2', argument1, option1


# Equivalent to @click.group() with special group
@click.command(cls=GroupExt)
@click.option('--optparam', default=None, type=str)
def group2():
    print 'in group2'


@group2.command()
def command2(optparam):
    print 'command2', optparam


@click.command(cls=click.CommandCollection, sources=[group1, group2])
def cli():
    print 'cli'


if __name__ == '__main__':
    cli(obj={})

This is not quite what I am looking for. Ideally, the optparam would be handled by group2 and the results placed into the context, but currently it's processed in the command2. Perhaps someone knows how to extend this.

Whim