Better usage of `make_pass_decorator` in Python Click

Question

I am looking for some advice to avoid having to instantiate a class twice; this is more of a design pattern question. I am creating an application using the Python Click library.

I have a Settings class that first loads all initial default settings into a dictionary (hard-coded into the application), then loads all settings overrides (if specified) from a TOML file on the user's computer into a dictionary, and then finally merges the two and makes them available as attributes of the class instance (settings.<something>).

For most of these settings, I also want to be able to specify a command-line flag. The priority then becomes:

1. Command-line flag. If not specified, then fallback to...
2. User setting in TOML file. If not specified, then finally fallback to...
3. Application default

In order to achieve this result, I am finding that, when using Click's decorators, I have to do something like this:

import click
from myapp import Settings

settings = Settings()
pass_settings = click.make_pass_decorator(Settings, ensure=True)

@click.command()
@click.help_option('-h', '--help')
@click.option(
    '-s', '--disk-size',
    default=settings.instance_disk_size,
    help="Disk size",
    show_default=True,
    type=int
)
@click.option(
    '-t', '--disk-type',
    default=settings.instance_disk_type,
    help="Disk type",
    show_default=True,
    type=click.Choice(['pd-standard', 'pd-ssd'])
)
@pass_settings
def create(settings, disk_size, disk_type):
    print(disk_size)
    print(disk_type)

Why twice?

• The settings = Settings() line is needed to provide the @click.option decorators with the default value. The default value could either come from the user override TOML file (if present), or from the application default.
• The click.make_pass_decorator seems to be the recommended way for interleaved commands; it's even mentioned in their documentation. Inside of the function, in addition to the CLI parameters passed, I also sometimes needs to reference other attributes in the Settings class.

My question is, which is better? Is there a way to use the pass_settings decorator in the other click.option decorators? Or should I ditch using click.make_pass_decorator entirely?

Mola

Answer 1

One way to approach the problem of not wanting to instantiate Settings twice, is to inherit from click.Option, and insert the settings instance into the context directly like:

Custom Class:

def build_settings_option_class(settings_instance):

    def set_default(default_name):

        class Cls(click.Option):
            def __init__(self, *args, **kwargs):
                kwargs['default'] = getattr(settings_instance, default_name)
                super(Cls, self).__init__(*args, **kwargs)

            def handle_parse_result(self, ctx, opts, args):
                obj = ctx.find_object(type(settings_instance))
                if obj is None:
                    ctx.obj = settings_instance

                return super(Cls, self).handle_parse_result(ctx, opts, args)

        return Cls

    return set_default
    

Using Custom Class:

To use the custom class, pass the cls parameter to @click.option() decorator like:

# instantiate settings
settings = Settings()

# get the setting option builder
settings_option_cls = build_settings_option_class(settings)

# decorate with an option with an appropraie option name
@click.option("--an_option", cls=settings_option_cls('default_setting_name'))

How does this work?

This works because click is a well designed OO framework. The @click.option() decorator usually instantiates a click.Option object but allows this behavior to be over ridden with the cls parameter. So it is a relatively easy matter to inherit from click.Option in our own class and over ride the desired methods.

In this case we use a couple of closures to capture the Settings instance and parameter name. In the returned class we over ride click.Option.handle_parse_result() to allow us to insert the setting object into the context. This allows the pass_settings decorator to find the settings in the context, and thus it will not need to create a new instance.

Test Code:

import click

class Settings(object):

    def __init__(self):
        self.instance_disk_size = 100
        self.instance_disk_type = 'pd-ssd'


settings = Settings()
settings_option_cls = build_settings_option_class(settings)
pass_settings = click.make_pass_decorator(Settings)


@click.command()
@click.help_option('-h', '--help')
@click.option(
    '-s', '--disk-size',
    cls=settings_option_cls('instance_disk_size'),
    help="Disk size",
    show_default=True,
    type=int
)
@click.option(
    '-t', '--disk-type',
    cls=settings_option_cls('instance_disk_type'),
    help="Disk type",
    show_default=True,
    type=click.Choice(['pd-standard', 'pd-ssd'])
)
@pass_settings
def create(settings, disk_size, disk_type):
    print(disk_size)
    print(disk_type)


if __name__ == "__main__":
    commands = (
        '-t pd-standard -s 200',
        '-t pd-standard',
        '-s 200',
        '',
        '--help',
    )

    import sys, time

    time.sleep(1)
    print('Click Version: {}'.format(click.__version__))
    print('Python Version: {}'.format(sys.version))
    for cmd in commands:
        try:
            time.sleep(0.1)
            print('-----------')
            print('> ' + cmd)
            time.sleep(0.1)
            create(cmd.split())

        except BaseException as exc:
            if str(exc) != '0' and \
                    not isinstance(exc, (click.ClickException, SystemExit)):
                raise
                

Test Results:

Click Version: 6.7
Python Version: 3.6.2 (default, Jul 17 2017, 23:14:31) 
[GCC 5.4.0 20160609]
-----------
> -t pd-standard -s 200
200
pd-standard
-----------
> -t pd-standard
100
pd-standard
-----------
> -s 200
200
pd-ssd
-----------
> 
100
pd-ssd
-----------
> --help
Usage: test.py [OPTIONS]

Options:
  -h, --help                      Show this message and exit.
  -s, --disk-size INTEGER         Disk size  [default: 100]
  -t, --disk-type [pd-standard|pd-ssd]
                                  Disk type  [default: pd-ssd]

Smallman

Stephen, such a high quality answer, and exactly what I was looking for! I'm getting better at Python, but questions like I had are often troublesome for a beginner/intermediate. Thank you so much! – Mola

@ScottCrooks, thanks for the kind words. I would reciprocate by saying your question was also very high quality. I have answered a fair number of Click Questions over the last year, and most are not nearly as well formulated as this one. A great question is often much easier to answer... Cheers and good luck on your Python journey. – Smallman

Answer 2

Differing opinion

Instead of modifying the click invocation and using dynamic class construction, expose the default settings as a class attribute for the Settings class. IE:

@click.option(
    '-t', '--disk-type',
    default=settings.instance_disk_type,
    help="Disk type",
    show_default=True,
    type=click.Choice(['pd-standard', 'pd-ssd'])
)

becomes

@click.option(
    '-t', '--disk-type',
    default=Settings.defaults.instance_disk_type,
    help="Disk type",
    show_default=True,
    type=click.Choice(['pd-standard', 'pd-ssd'])
)

This is likely cleaner and makes the semantics (meaning) of your code much clearer than using a class constructor s in the accepted answer.

In fact, the Settings.defaults could well be an instance of Settings. It doesn't mater that you're instantiating twice, as this isn't really the issue here, rather that your client/consumer code for the Settings object has to perform the instantiation. If that's done in the Settings class, it remains a clean API and doesn't require the caller to instantiate twice.

Psilocybin