Best practices for writing argparse parsers

Question

Are there best practices or style guidelines for working with Python's argparse module?

I work with argparse on a regular basis, and it quickly takes up a respectable number of lines to handle all the configuration. For almost everything I find that sticking close to PEP 8 results in clean, readable code, but not here. The end result is always an ugly block of code that is painful to read.

Painful to read is not Pythonic:

Beautiful is better than ugly ... Readibilty counts

So is there a PEP or some other resource that provides guidelines for how to better format this code?

A sample of the ugliness (mostly following PEP 8):

parser = argparse.ArgumentParser(description='A nontrivial modular command')
subparsers = parser.add_subparsers(help='sub-command help')

parser_load = subparsers.add_parser('load', help='Load something somewhere')
parser_load.add_argument('--config',
                         help='Path to configuration file for special settings')
parser_load.add_argument('--dir', default=os.getcwd(),
                         help='The directory to load')
parser_load.add_argument('book', help='The book to load into this big thing')
parser_load.add_argument('chapter', nargs='?', default='',
                         help='Optionally specify a chapter')
parser_load.add_argument('verse', nargs='*',
                         help='Optionally pick as many verses as you want to'
                         ' load')
parser_load.set_defaults(command='load')

parser_write = subparsers.add_parser(
                'write', help='Execute commands defined in a config file')
parser_write.add_argument('config', help='The path to the config file')
parser_write.set_defaults(command='write')

parser_save = subparsers.add_parser(
                'save',
                help='Save this big thing for use somewhere later')
parser_save.add_argument('-n', '--name', default=None,
                         help='The name of the component to save')
parser_save.add_argument('path', help="The way out of Plato's cave")
parser_save.set_defaults(command='save')

...

args = parser.parse_args()

Liman

check out click: click.pocoo.org/5, much nicer arguments via decorators – Narcose

Aside from not being consistent about where you're breaking lines I don't see any problems: You appear to be only breaking lines where it's necessary to conform to 80 characters, rather than where it aids readability. I'm not convinced this isn't off-topic: opinion based, as it's going to vary depending on your coding standards. – Hashish

@Hashish I can see that, that's why I asked if standards exist, rather than just suggestions for how to format this code – Liman

Maybe also rename parser_X/parser_Y to X_parser/Y_parser so that not every line begins with parser and X/Y can be distinguished more easily. – Rosenthal

I tend to use required=False in lieu of nargs='?'. If required is false, the value is none if it's not specified, and it makes my code easier to read because I know that nargs means "get ready to do something with a list". – Delossantos

@onwsk8r, required is not allowed as a parameter for a positional (try it). It automatically sets the required attribute based on nargs. required=True may be useful for a optional (a 'required optional'?). nargs='?' is most useful when used along with a const and default parameter (also optionals). – Panay

@hpaulj, thanks for pointing that out; I missed the lack of --. Maybe @Rosenthal was on to something :) I usually use comments to separate parsers as well, with positional arguments at the top. FWIW, in Go, I'll use spf13/cobra and have those subparsers in their own functions - much more intuitive. – Delossantos

Answer 1

As commented by TemporalWolf, I would use line breaks more consistently, and more of them. Even if the code now appears longer, I find it easier to read:

• More vertical space between individual function calls, therefore easier to distinguish visually
• One argument per line, therefore easier to see which ones are used
• Arguments closer to the left margin, therefore less horizontal eye movement and fewer unwanted line breaks (like the one where you split the help string) required

Additionally, by renaming parser_X/parser_Y → X_parser/Y_parser you could make it easier to distinguish X/Y.

parser = argparse.ArgumentParser(
    description='A nontrivial modular command'
)
subparsers = parser.add_subparsers(
    help='sub-command help'
)

load_parser = subparsers.add_parser(
    'load',
    help='Load something somewhere'
)
load_parser.add_argument(
    '--config',
    help='Path to configuration file for special settings'
)
load_parser.add_argument(
    '--dir',
    default=os.getcwd(),
    help='The directory to load'
)
load_parser.add_argument(
    'book',
    help='The book to load into this big thing'
)
load_parser.add_argument(
    'chapter',
    nargs='?',
    default='',
    help='Optionally specify a chapter'
)
load_parser.add_argument(
    'verse',
    nargs='*',
    help='Optionally pick as many verses as you want to load'
)
load_parser.set_defaults(
    command='load'
)

write_parser = subparsers.add_parser(
    'write',
    help='Execute commands defined in a config file'
)
write_parser.add_argument(
    'config',
    help='The path to the config file'
)
write_parser.set_defaults(
    command='write'
)

save_parser = subparsers.add_parser(
    'save',
    help='Save this big thing for use somewhere later'
)
save_parser.add_argument(
    '-n', '--name',
    default=None,
    help='The name of the component to save'
)
save_parser.add_argument(
    'path',
    help="The way out of Plato's cave"
)
save_parser.set_defaults(
    command='save'
)

...

args = parser.parse_args()

Rosenthal

Not sure why you didn't implement the naming change you mentioned in your comment. I 100% agree with it. My preferred formatting is not that dissimilar, but with your suggestion I find this even better and likely allows for removing the comments. – Hashish

@Hashish I didn't implement it (yet) because I had the idea only after posting the answer and didn't feel like editing everything again. – Rosenthal

Answer 2

There is nothing wrong with your code, that's just the consequence of using the argparse module. My personal preference is to break up the creation of the parser into functions. In this case, you can create a function for each subparser you create.

def parse_args(*args):
    parser = argparse.ArgumentParser(description='A nontrivial modular command')
    subparsers = parser.add_subparsers(help='sub-command help')

    add_load_subparser(subparsers)
    add_write_subparser(subparsers)
    add_save_subparser(subparsers)

    return parser.parse_args(*args)


def add_load_subparser(subparsers):
    parser = subparsers.add_parser('load', help='Load something somewhere')
    parser.add_argument('--config',
                        help='Path to configuration file for special settings')
    parser.add_argument('--dir', default=os.getcwd(),
                        help='The directory to load')
    parser.add_argument('book', help='The book to load into this big thing')
    parser.add_argument('chapter', nargs='?', default='',
                        help='Optionally specify a chapter')
    parser.add_argument('verse', nargs='*',
                        help='Optionally pick as many verses as you want to'
                        ' load')
    parser.set_defaults(command='load')


def add_write_subparser(subparsers):
    parser = subparsers.add_parser(
          'write', help='Execute commands defined in a config file')
    parser.add_argument('config', help='The path to the config file')
    parser.set_defaults(command='write')


def add_save_subparser(subparsers):
    parser = subparsers.add_parser(
               'save',
               help='Save this big thing for use somewhere later')
    parser.add_argument('-n', '--name', default=None,
                        help='The name of the component to save')
    parser.add_argument('path', help="The way out of Plato's cave")
    parser.set_defaults(command='save')


args = parse_args()

Talithatalk

Answer 3

There's not been any discussion about style for this particular module among developers (I've been following the relevant bug/issues closely).

I'm more concerned with solving problems than with style and layout, but do like code that's easy to read and understand. If there are large blocks of repeated patterns I like to use utility functions, dictionaries and lists.

A recent SO question, How to design object oriented subparsers for argparse? asked about OOP subparser definitions. I took his initial class and added a method:

  def make_sup(self,sp):
      self.parser = sp.add_parser(self.name)
      self.parser.add_argument('--foo')
      self.parser.set_defaults(action=self)

So a set of objects could be defined with

cmds = []
cmds.append(Cmd('list'))
cmds.append(Cmd('foo'))
cmds.append(Cmd('bar'))

or even

cmds = [Cmd('list'), Cmd('foo'),...]

and then used to populate the parser with:

parser = argparse.ArgumentParser()
sp = parser.add_subparsers(dest='cmd')
for cmd in cmds:
    cmd.make_sup(sp)

This is simple example that doesn't involve arguments.

The unittest file, test_argparse.py has a rather elaborate system to streamline parser defintions.

class Sig(object):

    def __init__(self, *args, **kwargs):
        self.args = args
        self.kwargs = kwargs

Test cases create lists of these Sig objects:

argument_signatures = [Sig('--foo-bar'), Sig('--baz', dest='zabbaz')]
argument_signatures = [
    Sig('-x', type=float),
    Sig('-3', type=float, dest='y'),
    Sig('z', nargs='*'),
]

And parser test class has methods like:

    def no_groups(parser, argument_signatures):
        """Add all arguments directly to the parser"""
        for sig in argument_signatures:
            parser.add_argument(*sig.args, **sig.kwargs)

Ipython has (or at least had a few versions back) code that creates a large argparse parser using config file entries to define the arguments.

Panay