Python Click - Supply arguments and options from a configuration file

Question

Given the following program:

#!/usr/bin/env python
import click

@click.command()
@click.argument("arg")
@click.option("--opt")
@click.option("--config_file", type=click.Path())
def main(arg, opt, config_file):
    print("arg: {}".format(arg))
    print("opt: {}".format(opt))
    print("config_file: {}".format(config_file))
    return

if __name__ == "__main__":
    main()

I can run it with the arguments and options provided through command line.

$ ./click_test.py my_arg --config_file my_config_file
arg: my_arg
opt: None
config_file: my_config_file

How do I provide a configuration file (in ini? yaml? py? json?) to --config_file and accept the content as the value for the arguments and options?

For instance, I want my_config_file to contain

opt: my_opt

and have the output of the program show:

$ ./click_test.py my_arg --config_file my_config_file
arg: my_arg
opt: my_opt
config_file: my_config_file

I've found the callback function which looked to be useful but I couldn't find a way to modify the sibling arguments/options to the same function.

Dekameter

Have you tried: github.com/jenisys/click-configfile? – Chalcography

I was hoping that this can be achieved without an external package – Dekameter

Answer 1

I realize that this is way old, but since Click 2.0, there's a more simple solution. The following is a slight modification of the example from the docs.

This example takes explicit --port args, it'll take an environment variable, or a config file (with that precedence).

Command Groups

Our code:

import os
import click
from yaml import load
try:
    from yaml import CLoader as Loader
except ImportError:
    from yaml import Loader


@click.group(context_settings={'auto_envvar_prefix': 'FOOP'})  # this allows for environment variables
@click.option('--config', default='~/config.yml', type=click.Path())  # this allows us to change config path
@click.pass_context
def foop(ctx, config):
    if os.path.exists(config):
        with open(config, 'r') as f:
            config = load(f.read(), Loader=Loader)
        ctx.default_map = config


@foop.command()
@click.option('--port', default=8000)
def runserver(port):
    click.echo(f"Serving on http://127.0.0.1:{port}/")


if __name__ == '__main__':
    foop()

Assuming our config file (~/config.yml) looks like:

runserver:
    port: 5000

and we have a second config file (at ~/config2.yml) that looks like:

runserver:
    port: 9000

Then if we call it from bash:

$ foop runserver
# ==> Serving on http://127.0.0.1:5000/
$ FOOP_RUNSERVER_PORT=23 foop runserver
# ==> Serving on http://127.0.0.1:23/
$ FOOP_RUNSERVER_PORT=23 foop runserver --port 34
# ==> Serving on http://127.0.0.1:34/
$ foop --config ~/config2.yml runserver
# ==> Serving on http://127.0.0.1:9000/

Single Commands

If you don't want to use command groups and want to have configs for a single command:

import os
import click
from yaml import load
try:
    from yaml import CLoader as Loader
except ImportError:
    from yaml import Loader


def set_default(ctx, param, value):
    if os.path.exists(value):
        with open(value, 'r') as f:
            config = load(f.read(), Loader=Loader)
        ctx.default_map = config
    return value


@click.command(context_settings={'auto_envvar_prefix': 'FOOP'})
@click.option('--config', default='config.yml', type=click.Path(),
              callback=set_default, is_eager=True, expose_value=False)
@click.option('--port')
def foop(port):
    click.echo(f"Serving on http://127.0.0.1:{port}/")

will give similar behavior.

Transceiver

Answer 2

This can be done by over riding the click.Command.invoke() method like:

Custom Class:

def CommandWithConfigFile(config_file_param_name):

    class CustomCommandClass(click.Command):

        def invoke(self, ctx):
            config_file = ctx.params[config_file_param_name]
            if config_file is not None:
                with open(config_file) as f:
                    config_data = yaml.safe_load(f)
                    for param, value in ctx.params.items():
                        if value is None and param in config_data:
                            ctx.params[param] = config_data[param]

            return super(CustomCommandClass, self).invoke(ctx)

    return CustomCommandClass

Using Custom Class:

Then to use the custom class, pass it as the cls argument to the command decorator like:

@click.command(cls=CommandWithConfigFile('config_file'))
@click.argument("arg")
@click.option("--opt")
@click.option("--config_file", type=click.Path())
def main(arg, opt, config_file):

Test Code:

# !/usr/bin/env python
import click
import yaml

@click.command(cls=CommandWithConfigFile('config_file'))
@click.argument("arg")
@click.option("--opt")
@click.option("--config_file", type=click.Path())
def main(arg, opt, config_file):
    print("arg: {}".format(arg))
    print("opt: {}".format(opt))
    print("config_file: {}".format(config_file))


main('my_arg --config_file config_file'.split())

Test Results:

arg: my_arg
opt: my_opt
config_file: config_file

Chalcography

Any opinions on how this compares to either click-config or click-configfile? – Cheeky

@eric.frederich, this was just a quick exercise to demonstrate how you could build your own thing, so no opinion pro/con vs libraries. – Chalcography

I think ctx.get_parameter_source(param) == ParameterSource.DEFAULT should be used instead of checking for None in the loop. – Gahl