How can I use click to parse a string for arguments?

Question

Say I have a list of strings containing arguments and options, with argparse, I’m able to parse this list using the parse_args function into an object, as follows:

import argparse

extra_params = [‘—sum’, ‘7’, ‘-1’, ‘42’]

parser=argparse.ArgumentParser(description=“argparse docs example”)
parser.add_argument(‘integers’, metavar=‘N’, type=int, nargs=‘+’,
                    help=‘an integer for the accumulator’)
parser.add_argument(‘—sum’, dest=‘accumulate’, action=‘store_const’,
                    const=sum, default=max,
                    help=‘sum the integers (default: find the max)’)
parsed_object=parser.parse_args(extra_params)

Here, argparse has parsed a provided iterable of strings. Can one use click to also parse a provided iterable of strings?

I’ve searched through the API documentation for click and it appears that there’s a parse_args function within the *Command set of classes but don’t see anything in the docs around how I can do this. I’ve tried instantiating BaseCommand as well as Command but not sure how to get parse_args working without a correct context.

For broader context, this question is a result of having built a launcher application that end users use as a scaffold to launch their own applications. Here, the launcher consumes a number of arguments for which click decorators work perfectly. Unknown arguments can be handled as shown in the documentation here. This launcher then calls an end-user provided callable with these unparsed parameters. Click leaves unparsed parameters as a tuple of strings. How would the end-user, in this situation, be able to use Click to parse the argument's they're interested in? Here's a snippet to illustrate the issue:

import click
from typing import Tuple

@click.command(name="TestLauncher", context_settings={
  "ignore_unknown_options": True
})
@click.option('--uri', '-u',
  help="URI for the server")
@click.argument('unprocessed_args', nargs=-1,
  type=click.UNPROCESSED)
def main(uri: str, unprocessed_args: Tuple[str, ...]) -> None:
    print(f"Was passed a URI of {uri}")
    print(f"Additional args are {unprocessed_args}")

    child_function(unprocessed_args)

def child_function(unprocessed_args: Tuple[str, ...]) -> None:
    # How do I get Click to parse the provided args for me?
    pass

if __name__ == "__main__":
    # pylint: disable=no-value-for-parameter, unexpected-keyword-arg
    main()

Running this from the command line:

python3 so_test.py --uri test.com --prog-arg 10
Was passed a URI of test.com
Additional args are ('--prog-arg', '10')

Percentage

But, one significant point of click, is not to build the amorous blob of a param definitions, and instead annotate functions with the params they handle. So in the context of click, I don't think your question makes any sense. What is the use case you are trying to deal with? And in that use case, why use click and not argparse, since you seem to already be comfortable with that? – Crittenden

What you are doing with extra_params is simulating the input normally read from the commandline (via sys.argv[1:]). This is most commonly use for testing (I use it frequently in SO answers). parsed_object or its equivalent could also be constructed directly. – Asia

@StephenRauch the use case is where I have a main programme that consumes a number of arguments/options. Click works perfectly here. However this main programme is a reusable launcher that others use as scaffolding for their own programmes. These child processes can accept an unknown number of arguments/options, so what is desired is for a way to pass unknown arguments/options to this child, done here and then have the child parse these arguments. Let me update the question with this context. – Percentage

Answer 1

Reviewing the comments and my ensuing edit, made me think that simply applying the click decorators to the child function may work. Indeed it seems to but I don't entirely know why.

import click
from typing import Tuple

@click.command(name="TestLauncher", context_settings={
  "ignore_unknown_options": True
})
@click.option('--uri', '-u',
  help="URI for the server")
@click.argument('unprocessed_args', nargs=-1,
  type=click.UNPROCESSED)
def main(uri: str, unprocessed_args: Tuple[str, ...]) -> None:
    print(f"Was passed a URI of {uri}")
    print(f"Additional args are {unprocessed_args}")

    child_function(unprocessed_args)

@click.command()
@click.option('--prog-arg')
def child_function(prog_arg: str) -> None:
    # How do I get Click to parse the provided args for me?
    print(f"Child function passed: {prog_arg}")

if __name__ == "__main__":
    # pylint: disable=no-value-for-parameter, unexpected-keyword-arg
    main()

python3 so_test.py --uri test.com --prog-arg 10
Was passed a URI of test.com
Additional args are ('--prog-arg', '10')
Child function passed: 10

Percentage

The reason that works is because you have turned child function into a click command, which your main method invokes. However, it's not advisable to directly call a click command from another one, as it can lead to some unexpected behaviour. For that end, you might want to look into Click's documentation for invoking a other commands click.palletsprojects.com/en/7.x/advanced/… – Dravidian

@Dravidian thanks for the pointer. One issue here is that ctx.invoke requires parsing the argument string to specify the keyword args which requires a priori knowledge of the child function's arguments (which isn't possible here). ctx.forward works however that parses arguments/options from the callers and passes them as kwargs to the receiver. In the above example, uri is a required kwarg for child_function as is unprocessed_args. This then leaves the child function with the task of parsing unprocessed_args which gets us back to step 1 with parsing an iterable of strings. – Percentage

Also @Dravidian do you have a sample snippet solving this where the calling function knows nothing about the parameters of the child function? Would also love to understand what unexpected behaviour might occur with the approach cited in the answer. – Percentage

I've added a quick comment, let me know if it helps – Dravidian

Answer 2

For the calling function not knowing anything about parameters for the child function, you can try this:

@click.command(name="TestLauncher", context_settings={
    "ignore_unknown_options": True
})
@click.option('--uri', '-u',
              help="URI for the server")
@click.argument('unprocessed_args', nargs=-1,
                type=click.UNPROCESSED)
def main(uri: str, unprocessed_args: Tuple[str, ...]) -> None:
    print(f"Was passed a URI of {uri}")
    print(f"Additional args are {unprocessed_args}")
    unprocessed_args = dict([(unprocessed_args[i].replace('--', '').replace('-', '_'), unprocessed_args[i+1]) for i in range(0, len(unprocessed_args), 2)])
    click.get_current_context().invoke(child_function, **unprocessed_args)

@click.command(context_settings={"ignore_unknown_options": True})
@click.option('-p', '--prog-arg')
def child_function(prog_arg: str, **kwargs) -> None:
    # How do I get Click to parse the provided args for me?
    print(f"Child function passed: {prog_arg}")
    # all remaining unknown options are in **kwargs

if __name__ == "__main__":
    # pylint: disable=no-value-for-parameter, unexpected-keyword-arg
    main()

However, note that:

unprocessed_args = dict([(unprocessed_args[i].replace('--', '').replace('-', '_'), unprocessed_args[i+1]) for i in range(0, len(unprocessed_args), 2)])

This assumes you can only have one value per option. The alternative is to call your script by passing in options like below, splitting the string on = and doing whatever pre-formatting you deem necessary.

--prog-arg=<Your-desired-values>

Dravidian

Answer 3

Try something like this:

import click

@click.command()
@click.option('--count', default=1, help='number of greetings')
@click.option('--test', default='test_was_not_provided', help='test option')
@click.argument('name')
def hello(*args, **kwargs):
    click.echo(f"Hello World! {kwargs['name']} {kwargs['count']}")


if __name__ == '__main__':
    hello()

run with something like: python main.py haha --test this_is_a_test --count=40

Acus