nargs=* equivalent for options in Click
Asked Answered
M

4

38

Is there an equivalent to argparse's nargs='*' functionality for optional arguments in Click?

I am writing a command line script, and one of the options needs to be able to take an unlimited number of arguments, like:

foo --users alice bob charlie --bar baz

So users would be ['alice', 'bob', 'charlie'] and bar would be 'baz'.

In argparse, I can specify multiple optional arguments to collect all of the arguments that follow them by setting nargs='*'.

>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--users', nargs='*')
>>> parser.add_argument('--bar')
>>> parser.parse_args('--users alice bob charlie --bar baz'.split())
Namespace(bar='baz', users=['alice', 'bob', 'charlie'])

I know Click allows you to specify an argument to accept unlimited inputs by setting nargs=-1, but when I try to set an optional argument's nargs to -1, I get:

TypeError: Options cannot have nargs < 0

Is there a way to make Click accept an unspecified number of arguments for an option?

Update:

I need to be able to specify options after the option that takes unlimited arguments.

Update:

@Stephen Rauch's answer answers this question. However, I don't recommend using the approach I ask for here. My feature request is intentionally not implemented in Click, since it can result in unexpected behaviors. Click's recommended approach is to use multiple=True:

@click.option('-u', '--user', 'users', multiple=True)

And in the command line, it will look like:

foo -u alice -u bob -u charlie --bar baz
Mendiola answered 22/1, 2018 at 23:16 Comment(2)
@gnarlyninja I was not able to replicate this with click 8.1. Does another setting need to be enabled for this to work?Mendiola
no sorry. I had a weird corner case where it was working but doesn't work across the board and I forgot to remove/edit my commentScarbrough
C
35

One way to approach what you are after is to inherit from click.Option, and customize the parser.

Custom Class:

import click

class OptionEatAll(click.Option):

    def __init__(self, *args, **kwargs):
        self.save_other_options = kwargs.pop('save_other_options', True)
        nargs = kwargs.pop('nargs', -1)
        assert nargs == -1, 'nargs, if set, must be -1 not {}'.format(nargs)
        super(OptionEatAll, self).__init__(*args, **kwargs)
        self._previous_parser_process = None
        self._eat_all_parser = None

    def add_to_parser(self, parser, ctx):

        def parser_process(value, state):
            # method to hook to the parser.process
            done = False
            value = [value]
            if self.save_other_options:
                # grab everything up to the next option
                while state.rargs and not done:
                    for prefix in self._eat_all_parser.prefixes:
                        if state.rargs[0].startswith(prefix):
                            done = True
                    if not done:
                        value.append(state.rargs.pop(0))
            else:
                # grab everything remaining
                value += state.rargs
                state.rargs[:] = []
            value = tuple(value)

            # call the actual process
            self._previous_parser_process(value, state)

        retval = super(OptionEatAll, self).add_to_parser(parser, ctx)
        for name in self.opts:
            our_parser = parser._long_opt.get(name) or parser._short_opt.get(name)
            if our_parser:
                self._eat_all_parser = our_parser
                self._previous_parser_process = our_parser.process
                our_parser.process = parser_process
                break
        return retval

Using Custom Class:

To use the custom class, pass the cls parameter to @click.option() decorator like:

@click.option("--an_option", cls=OptionEatAll)

or if it is desired that the option will eat the entire rest of the command line, not respecting other options:

@click.option("--an_option", cls=OptionEatAll, save_other_options=False)

How does this work?

This works because click is a well designed OO framework. The @click.option() decorator usually instantiates a click.Option object but allows this behavior to be over ridden with the cls parameter. So it is a relatively easy matter to inherit from click.Option in our own class and over ride the desired methods.

In this case we over ride click.Option.add_to_parser() and the monkey patch the parser so that we can eat more than one token if desired.

Test Code:

@click.command()
@click.option('-g', 'greedy', cls=OptionEatAll, save_other_options=False)
@click.option('--polite', cls=OptionEatAll)
@click.option('--other')
def foo(polite, greedy, other):
    click.echo('greedy: {}'.format(greedy))
    click.echo('polite: {}'.format(polite))
    click.echo('other: {}'.format(other))


if __name__ == "__main__":
    commands = (
        '-g a b --polite x',
        '-g a --polite x y --other o',
        '--polite x y --other o',
        '--polite x -g a b c --other o',
        '--polite x --other o -g a b c',
        '-g a b c',
        '-g a',
        '-g',
        'extra',
        '--help',
    )

    import sys, time
    time.sleep(1)
    print('Click Version: {}'.format(click.__version__))
    print('Python Version: {}'.format(sys.version))
    for cmd in commands:
        try:
            time.sleep(0.1)
            print('-----------')
            print('> ' + cmd)
            time.sleep(0.1)
            foo(cmd.split())

        except BaseException as exc:
            if str(exc) != '0' and \
                    not isinstance(exc, (click.ClickException, SystemExit)):
                raise

Test Results:

Click Version: 6.7
Python Version: 3.6.3 (v3.6.3:2c5fed8, Oct  3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)]
-----------
> -g a b --polite x
greedy: ('a', 'b', '--polite', 'x')
polite: None
other: None
-----------
> -g a --polite x y --other o
greedy: ('a', '--polite', 'x', 'y', '--other', 'o')
polite: None
other: None
-----------
> --polite x y --other o
greedy: None
polite: ('x', 'y')
other: o
-----------
> --polite x -g a b c --other o
greedy: ('a', 'b', 'c', '--other', 'o')
polite: ('x',)
other: None
-----------
> --polite x --other o -g a b c
greedy: ('a', 'b', 'c')
polite: ('x',)
other: o
-----------
> -g a b c
greedy: ('a', 'b', 'c')
polite: None
other: None
-----------
> -g a
greedy: ('a',)
polite: None
other: None
-----------
> -g
Error: -g option requires an argument
-----------
> extra
Usage: test.py [OPTIONS]

Error: Got unexpected extra argument (extra)
-----------
> --help
Usage: test.py [OPTIONS]

Options:
  -g TEXT
  --polite TEXT
  --other TEXT
  --help         Show this message and exit.
Calefacient answered 23/1, 2018 at 4:1 Comment(7)
This is wonderful. Thank you so much for taking the time to post this answer!Audiovisual
Seems like this is broken in Click v8Heaves
@KlemenTusar, I just reran the testcode with click 8.0.1 and got the exact same results as w/ 6.7 from above. Perhaps you can write up a question documenting what you think is broken?Calefacient
@StephenRauch I'm still investigating, to be honest, however, here's the GitHub issue of one of my packages that uses your solution above github.com/techouse/mysql-to-sqlite3/issues/15. Seems to work fine with Click 7.1.2 but not 8+. I'll keep digging and will report back :)Heaves
Seems that with Click v7.x it returns a tuple but with v8.x it returns a string. <class 'tuple'> ('content_files', 'content_media', 'content_usages', 'iframes') vs <class 'str'> ('content_files', 'content_media', 'content_usages', 'iframes')Heaves
Added a better description here in the Github issue github.com/techouse/mysql-to-sqlite3/issues/…Heaves
It seems it's related to this Click issue github.com/pallets/click/issues/2012 and simply defining declaring type=tuple on the click option fixes it.Heaves
A
12

You can use this trick.

import click

@click.command()
@click.option('--users', nargs=0, required=True)
@click.argument('users', nargs=-1)
@click.option('--bar')
def fancy_command(users, bar):
    users_str = ', '.join(users)
    print('Users: {}. Bar: {}'.format(users_str, bar))

if __name__ == '__main__':
    fancy_command()

Add fake option with a needed name and none arguments nargs=0, then add 'argument' with the unlimited args nargs=-1.

$ python foo --users alice bob charlie --bar baz
Users: alice, bob, charlie. Bar: baz

But be careful with the further options:

$ python foo --users alice bob charlie --bar baz faz
Users: alice, bob, charlie, faz. Bar: baz
Arbitrage answered 19/2, 2019 at 10:44 Comment(4)
But in this setup it will be possible to omit --users flag at all. Not sure if that is expected.Defendant
Thank you, @MarSoft! Missed that! Changed code above, adding required=True. Now it's not possible to omit ---users. Unfortunately, the flag string can be omitted. Same results: $ python foo --users alice bob charlie --bar baz faz and $ python foo alice bob charlie --bar baz fazArbitrage
This no longer works, I get Error: Invalid value for '--users': Takes 0 values but 3 were given. I am using Click 8.0.1.Zumstein
See my answer with a reasonable alternative that works: https://mcmap.net/q/406587/-nargs-equivalent-for-options-in-clickZumstein
B
5

I ran into the same issue. Instead of implementing a single command line option with n number of arguments, I decided to use multiple of the same command line option and just letting Click make a tuple out of the arguments under the hood. I ultimately figured if Click didn't support it, that decision was probably made for a good reason.

https://click.palletsprojects.com/en/7.x/options/#multiple-options

here is an example of what I am saying:

instead of passing a single string argument a splitting on a delimiter:

commit -m foo:bar:baz

I opted to use this:

commit -m foo -m bar -m baz

here is the source code:

@click.command()
@click.option('--message', '-m', multiple=True)
def commit(message):
    click.echo('\n'.join(message))

This is more to type, but I do think it makes the CLI more user friendly and robust.

Bullring answered 21/1, 2020 at 20:3 Comment(2)
This is the right answer for the interface, and the one I ultimately chose. nargs=+ is a nice hack, but it typically encourages bad CLI designMendiola
Thank you for your answer! I really believe that this is the correct way to use click for this behavior. Thanks for taking the time to write this answer out and post it here.Seligman
Z
2

I needed this for myself and thought of settling for the solution provided by @nikita-malovichko even though it is very restrictive, but it didn't work for me (see my comment to that answer) so came up with the below alternative.

My solution doesn't directly address the question on how to support nargs=*, but it provided a good alternative for myself so sharing it for the benefit of others.

The idea is to use one option that specifies the expected count for another, i.e., set the nargs count dynamically at runtime. Here is a quick demo:

import click


def with_dynamic_narg(cnt_opt, tgt_opt):
    class DynamicNArgSetter(click.Command):
        def parse_args(self, ctx, args):
            ctx.resilient_parsing = True
            parser = self.make_parser(ctx)
            opts, _, _ = parser.parse_args(args=list(args))
            if cnt_opt in opts:
                for p in self.params:
                    if isinstance(p, click.Option) and p.name == tgt_opt:
                        p.nargs = int(opts[cnt_opt])

            ctx.resilient_parsing = False
            return super().parse_args(ctx, args)

    return DynamicNArgSetter


@click.command(cls=with_dynamic_narg('c', 'n'))
@click.option("-c", type=click.INT)
@click.option("-n", nargs=0)
def f(c, n):
    print(c, n)

if __name__ == '__main__':
    f()

In the above code, a custom Command class is created that knows the link between the "count" arg and the target arg that takes multiple args. It first does a local parsing in "resilient" mode to detect the count, then uses the count to update the nargs value of the target arg and then resumes parsing in normal mode.

Here is some sample interaction:

$ python t.py -c 0
0 None
$ python t.py -c 1
Usage: t.py [OPTIONS]
Try 't.py --help' for help.

Error: Missing option '-n'.
$ python t.py -c 0 -n a
Usage: t.py [OPTIONS]
Try 't.py --help' for help.

Error: Got unexpected extra argument (a)
$ python t.py -c 1 -n a
1 a
$ python /tmp/t.py -c 2 -n a b
2 ('a', 'b')

Note: The advantage over the official recommendation of using multiple=True is that we can use filename wildcards and let shell expand them. E.g.,

$ touch abc.1 abc.2
$ python t.py -c 2 -n abc.*
2 ('abc.1', 'abc.2')
$ python t.py -c $(echo abc.* | wc -w) -n abc.*
2 ('abc.1', 'abc.2')
Zumstein answered 29/1, 2023 at 12:52 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.