Argparse with required subparser

Question

I'm using Python 3.4, I'm trying to use argparse with subparsers, and I want to have a similar behavior to the one in Python 2.x where if I don't supply a positional argument (to indicate the subparser/subprogram) I'll get a helpful error message. I.e., with python2 I'll get the following error message:

$ python2 subparser_test.py    
usage: subparser_test.py [-h] {foo} ...
subparser_test.py: error: too few arguments

I'm setting the required attribute as suggested in https://mcmap.net/q/205325/-why-does-this-argparse-code-behave-differently-between-python-2-and-3, however that gives me an error with Python 3.4.0: TypeError: sequence item 0: expected str instance, NoneType found - full traceback:

$ python3 subparser_test.py    
Traceback (most recent call last):
  File "subparser_test.py", line 17, in <module>
    args = parser.parse_args()
  File "/usr/local/Cellar/python3/3.4.0/Frameworks/Python.framework/Versions/3.4/lib/python3.4/argparse.py", line 1717, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/local/Cellar/python3/3.4.0/Frameworks/Python.framework/Versions/3.4/lib/python3.4/argparse.py", line 1749, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/local/Cellar/python3/3.4.0/Frameworks/Python.framework/Versions/3.4/lib/python3.4/argparse.py", line 1984, in _parse_known_args
    ', '.join(required_actions))
TypeError: sequence item 0: expected str instance, NoneType found

This is my program subparser_test.py - adapted from https://docs.python.org/3.2/library/argparse.html#sub-commands:

import argparse

# sub-command functions
def foo(args):
    print('"foo()" called')

# create the top-level parser
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers()
subparsers.required = True

# create the parser for the "foo" command
parser_foo = subparsers.add_parser('foo')
parser_foo.set_defaults(func=foo)

args = parser.parse_args()
args.func(args)

Related question: Why does this argparse code behave differently between Python 2 and 3?

Clouded

Answer 1

You need to give subparsers a dest.

parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(dest='cmd')
subparsers.required = True

Now:

1909:~/mypy$ argdev/python3 stack23349349.py
usage: stack23349349.py [-h] {foo} ...
stack23349349.py: error: the following arguments are required: cmd

In order to issue this 'missing arguments' error message, the code needs to give that argument a name. For a positional argument (like subparses), that name is (by default) the 'dest'. There's a (minor) note about this in the SO answer you linked.

One of the few 'patches' to argparse in the last Python release changed how it tests for 'required' arguments. Unfortunately it introduced this bug regarding subparsers. This needs to be fixed in the next release (if not sooner).

update

If you want this optional subparsers behavior in Py2, it looks like the best option is to use a two stage parser as described in

How to Set a Default Subparser using Argparse Module with Python 2.7

There has been some recent activity in the related bug/issue

https://bugs.python.org/issue9253

update

A fix to this is in the works: https://github.com/python/cpython/pull/3027

Kerchief

More information about the argparse bug here. Upvoting that answer seems to help raise its urgency in the Python bug queue. Really, I'm not just farming rep for someone! – Jordans

Update: the fix has been reverted for "compatibility" reasons: bugs.python.org/issue33109 -- guess this bug will live on forever – Moulmein

Since Python 3.7 required can be used as argument: parser.add_subparsers(dest='cmd', required=True), docs.python.org – Chemarin

the dest parameter is no longer required, only required is :p. As in, you can just specify parser.add_subparsers(required=True) if you don't want to give it a named destination. – Entwistle

@Entwistle that gives an incomprehensible error message if the argument is missing. – Astronomy

@Astronomy parser.add_subparsers(metavar='command', required=True) works well, the help is OK... tested with python 3.10.5 and 3.6.8 (required=True must be subparsers.required = True in 3.6). – Spoilsport

The issue with all of these solutions is the help of the original parser is lost and not printed! – Jocose

Answer 2

The previous mentioned workaround is no longer required as the add_mutually_exclusive_group() method also accepts a required argument.

This will indicate that least one of the mutually exclusive arguments is required.

parser = argparse.ArgumentParser(prog='PROG')
group = parser.add_mutually_exclusive_group(required=True)

group.add_argument('--foo', action='store_true')
group.add_argument('--bar', action='store_false')

> parser.parse_args([])
> usage: PROG [-h] (--foo | --bar)
> PROG: error: one of the arguments --foo --bar is required

Taken from the argsparse docs: https://docs.python.org/3/library/argparse.html#argparse.ArgumentParser.add_mutually_exclusive_group

Aurlie

Answer 3

I've came up with a handy shim layer which basically backports parser.add_subparsers(required=True) added in Python 3.7 to Python 3.6 or older:

class ArgParseShim:
    def __init__(self):
        self.subparsers = []

    def add_required_subparsers(self, p):
        res = p.add_subparsers()
        self.subparsers.append(res)
        return res

    def epilogue(self):
        for sub in self.subparsers:
            keys = sub.choices.keys()
            assert len(keys) > 0, "required subparsers must have sub parser (mouthful it is!)"
            sub.metavar = '{' + ','.join(keys) + '}'
            sub.required = True

call shim.add_required_subparsers(p) as a substitute of p.add_subparsers(required=True) and call shim.epilogue() after all commands have been configured. usecase:

if __name__ == '__main__':
    shim = ArgParseShim()
    parser = argparse.ArgumentParser()
    subs = shim.add_required_subparsers(parser)
    parser_warehouse = subs.add_parser("warehouse")
    parser_warehouse.add_argument("--config", "-c")
    warehouse_subs = shim.add_required_subparsers(parser_warehouse)
    create_warehouse = warehouse_subs.add_parser("create")
    drop_warehouse = warehouse_subs.add_parser("drop")
    shim.epilogue()
    parser.parse_args()

    ctl = Controller()
    create_warehouse.set_defaults(func=ctl.create_warehouse) # use set_defaults to link commands to workers as recommended in https://docs.python.org/3.6/library/argparse.html#:~:text=One%20particularly%20effective%20way
    drop_warehouse.set_defaults(func=ctl.drop_warehouse)
    parser.parse_args().func()

Macaronic