argparse subparser monolithic help output

Question

My argparse has only 3 flags (store_true) on the top level, everything else is handled through subparsers. When I run myprog.py --help, the output shows a list of all subcommands like normal, {sub1, sub2, sub3, sub4, ...}. So, the default is working great...

I usually can't remember the exact subcommand name I need, and all of its options. So I end up doing 2 help lookups:

myprog.py --help
myprog.py sub1 --help

I do this so often, I decided to cram this into one step. I would rather have my toplevel help output a huge summary, and then I scroll through the list manually. I find it is much faster (for me at least).

I was using a RawDescriptionHelpFormatter, and typing the long help output by hand. But now I have lots of subcommands, and its becoming a pain to manage.

Is there a way to get a verbose help output with just one program call?

If not, how can I iterate the subparsers of my argparse instance, and then retrieve the help output individually from each one (which I will then later glue together)?

---

Here is a quick outline of my argparse setup. I cleaned/stripped the code a fair bit, so this may not run without a bit of help.

parser = argparse.ArgumentParser(
        prog='myprog.py',
        formatter_class=argparse.RawDescriptionHelpFormatter,
        description=textwrap.dedent(""" You can manually type Help here """) )

parser.add_argument('--debuglog', action='store_true', help='Verbose logging for debug purposes.')
parser.add_argument('--ipyonexit', action='store_true', help='Drop into an embeded Ipython session instead of exiting command.')

subparser = parser.add_subparsers()

### --- Subparser B
parser_b = subparser.add_parser('pdfreport', description="Used to output reports in PDF format.")
parser_b.add_argument('type', type=str, choices=['flatlist', 'nested', 'custom'],
                        help="The type of PDF report to generate.")
parser_b.add_argument('--of', type=str, default='',
                        help="Override the path/name of the output file.")
parser_b.add_argument('--pagesize', type=str, choices=['letter', '3x5', '5x7'], default='letter',
                        help="Override page size in output PDF.")
parser_b.set_defaults(func=cmd_pdf_report)

### ---- Subparser C
parser_c = subparser.add_parser('dbtables', description="Used to perform direct DB import/export using XLS files.")
parser_c.add_argument('action', type=str, choices=['push', 'pull', 'append', 'update'],
                        help="The action to perform on the Database Tables.")
parser_c.add_argument('tablename', nargs="+",
                        help="The name(s) of the DB-Table to operate on.")
parser_c.set_defaults(func=cmd_db_tables)

args = parser.parse_args()
args.func(args)

Zucchetto

Show us a tiny example with some code, just a couple options and a couple subparsers. – Embosom

Answer 1

This is a bit tricky, as argparse does not expose a list of defined sub-parsers directly. But it can be done:

import argparse

# create the top-level parser
parser = argparse.ArgumentParser(prog='PROG')
parser.add_argument('--foo', action='store_true', help='foo help')
subparsers = parser.add_subparsers(help='sub-command help')

# create the parser for the "a" command
parser_a = subparsers.add_parser('a', help='a help')
parser_a.add_argument('bar', type=int, help='bar help')

# create the parser for the "b" command
parser_b = subparsers.add_parser('b', help='b help')
parser_b.add_argument('--baz', choices='XYZ', help='baz help')
# print main help
print(parser.format_help())

# retrieve subparsers from parser
subparsers_actions = [
    action for action in parser._actions 
    if isinstance(action, argparse._SubParsersAction)]
# there will probably only be one subparser_action,
# but better safe than sorry
for subparsers_action in subparsers_actions:
    # get all subparsers and print help
    for choice, subparser in subparsers_action.choices.items():
        print("Subparser '{}'".format(choice))
        print(subparser.format_help())

This example should work for python 2.7 and python 3. The example parser is from Python 2.7 documentation on argparse sub-commands.

The only thing left to do is adding a new argument for the complete help, or replacing the built in -h/--help.

Population

Great example. This is generating good output for me. Im not sure how to redefine the -h/--help argument in my case, as optional args do not like to follow my subparsers. Although, I may just define another subparser, named 'help', as the very last, and it can inspect everything added prior to it. – Zucchetto

I added another subparser named help. This solution is great because I can turn it into a function that accepts 'parser' and nothing else. – Gulley

Why do not use hasattr() instead of list comprehension and for loops? – Antechamber

Answer 2

Here is complete soulution with custom help handler (almost all code from @Adaephon answer):

import argparse


class _HelpAction(argparse._HelpAction):

    def __call__(self, parser, namespace, values, option_string=None):
        parser.print_help()

        # retrieve subparsers from parser
        subparsers_actions = [
            action for action in parser._actions
            if isinstance(action, argparse._SubParsersAction)]
        # there will probably only be one subparser_action,
        # but better save than sorry
        for subparsers_action in subparsers_actions:
            # get all subparsers and print help
            for choice, subparser in subparsers_action.choices.items():
                print("Subparser '{}'".format(choice))
                print(subparser.format_help())

        parser.exit()

# create the top-level parser
parser = argparse.ArgumentParser(prog='PROG', add_help=False)  # here we turn off default help action

parser.add_argument('--help', action=_HelpAction, help='help for help if you need some help')  # add custom help

parser.add_argument('--foo', action='store_true', help='foo help')
subparsers = parser.add_subparsers(help='sub-command help')

# create the parser for the "a" command
parser_a = subparsers.add_parser('a', help='a help')
parser_a.add_argument('bar', type=int, help='bar help')

# create the parser for the "b" command
parser_b = subparsers.add_parser('b', help='b help')
parser_b.add_argument('--baz', choices='XYZ', help='baz help')

parsed_args = parser.parse_args()

Rennarennane

It's probably better to use parser.add_argument ('-h', '--help', action=_HelpAction, help='show this help message and exit') to match the default argparse --help option. – Canescent

how to make it print help when no arguments are provided? – Carat

@kanna, please see this answer: https://mcmap.net/q/98263/-display-help-message-with-python-argparse-when-script-is-called-without-any-arguments – Rennarennane

Answer 3

Perhaps an easier approach is to use parser.epilog:

def define_parser():
    import argparse
    parser = argparse.ArgumentParser(
        prog='main',
        formatter_class=argparse.RawDescriptionHelpFormatter,
    )
    commands = parser.add_subparsers(
        title="required commands",
        help='Select one of:',
    )    
    command_list = commands.add_parser(
        'list',
        help='List included services',
    )
    command_ensure = commands.add_parser(
        'ensure',
        help='Provision included service',
    )
    command_ensure.add_argument(
        "service",
        help='Service name',
    )
    import textwrap
    parser.epilog = textwrap.dedent(
        f"""\
        commands usage:\n
        {command_list.format_usage()}
        {command_ensure.format_usage()}
        """
    )
    return parser

parser = define_parser()

parser.print_help()

which results in the following output:

usage: main [-h] {list,ensure} ...

optional arguments:
  -h, --help     show this help message and exit

required commands:
  {list,ensure}  Select one of:
    list         List included services
    ensure       Provision included service

commands usage:

usage: main list [-h]

usage: main ensure [-h] service

Houseless

Not the greatest layout but love the simplicity. textwrap is not really necessary, and we can remove the extra newline by putting all usage on one line. – Palma

In fact, we can as well change the usage directly, taking care of stripping the prefix Usage: in the ouput: parser.usage=f"{parser.format_usage()[7:]}{command_list.format_usage()}{command_ensure.format_usage()}". This gives almost perfect layout. – Palma

Answer 4

A simpler way to iterate over the subparsers in Adaephon's example is

for subparser in [parser_a, parser_b]:
   subparser.format_help()

Python does allow you to access hidden attributes like parser._actions, but that's not encouraged. It is just as easy to build your own list while defining the parser. Same goes for doing special things with the arguments. add_argument and add_subparser return their respective Action and Parser objects for a reason.

If I were making a subclass of ArgumentParser I would feel free to use _actions. But for a one off application, building my own list would be clearer.

---

An example:

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('mainpos')
parser.add_argument('--mainopt')
sp = parser.add_subparsers()
splist = []   # list to collect subparsers
sp1 = sp.add_parser('cmd1')
splist.append(sp1)
sp1.add_argument('--sp1opt')
sp2 = sp.add_parser('cmd2')
splist.append(sp2)
sp2.add_argument('--sp2opt')

# collect and display for helps    
helps = []
helps.append(parser.format_help())
for p in splist:
   helps.append(p.format_help())
print('\n'.join(helps))

# or to show just the usage
helps = []
helps.append(parser.format_usage())
for p in splist:
   helps.append(p.format_usage())
print(''.join(helps))

The combined 'usage' display is:

usage: stack32607706.py [-h] [--mainopt MAINOPT] mainpos {cmd1,cmd2} ...
usage: stack32607706.py mainpos cmd1 [-h] [--sp1opt SP1OPT]
usage: stack32607706.py mainpos cmd2 [-h] [--sp2opt SP2OPT]

The display of the combined helps is long and redundant. It could be edited in various ways, either after formatting, or with special help formatters. But who is going make such choices?

Conchitaconchobar

Answer 5

add_subparsers().add_parser() accepts not only a description, which shows up in the help of the subcommand, but also a help= which is used as one-line description in the top-level parsers' help.

The docs have this hidden in the formulation

(A help message for each subparser command, however, can be given by supplying the help= argument to add_parser() as above.)

and even in the sample code around that sentence:

>>> # create the parser for the "b" command
>>> parser_b = subparsers.add_parser('b', help='b help')
>>> parser_b.add_argument('--baz', choices='XYZ', help='baz help')

[...]

usage: PROG [-h] [--foo] {a,b} ...

positional arguments:
  {a,b}   sub-command help
    a     a help
    b     b help

Yes, this is not the full help for everthing, but IMHO covers the basic use case very well and is not easily discoverable.

Coincident

Answer 6

I was also able to print a short help for commands using _choices_actions.

def print_help(parser):
  print(parser.description)
  print('\ncommands:\n')

  # retrieve subparsers from parser
  subparsers_actions = [
      action for action in parser._actions 
      if isinstance(action, argparse._SubParsersAction)]
  # there will probably only be one subparser_action,
  # but better save than sorry
  for subparsers_action in subparsers_actions:
      # get all subparsers and print help
      for choice in subparsers_action._choices_actions:
          print('    {:<19} {}'.format(choice.dest, choice.help))

Kristykristyn

Answer 7

if __name__ == '__main__':
parser = argparse.ArgumentParser("TOML FILE OVERWRITE SCRIPT")
subparsers = parser.add_subparsers()

parser_set = subparsers.add_parser('set', help='Set Toml')
parser_set.add_argument('set', help='TOMl file edit set action', action='store_true')
parser_set.add_argument('-n', '--name', type=str, help='Service Name', required=True)
parser_set.add_argument('-s', '--section', type=str, help='Toml Section Name', required=True)
parser_set.add_argument('-k', '--key', type=str, help='Toml Key of Section', required=True)
parser_set.add_argument('-v', '--value', help='New Value', required=True)
args = parser.parse_args()

if args.set:
    setter = ConfigurationSetter(args.name, args.section, args.key, args.value)
    setter.execute()
else:
    print("Ops! Something is wrong, type --help or -h")

You can check my code maybe inspires you!

Machree

Answer 8

Simple deep recursive monolithic help output that doesn't look terrible and is easy to customize:

def recursive_help(parser):
    parser.print_help()
    def remove_argument(parser, arg):
        for action in parser._actions:
            opts = action.option_strings
            if (opts and opts[0] == arg) or action.dest == arg:
                parser._remove_action(action)
                break

        for action in parser._action_groups:
            for group_action in action._group_actions:
                opts = group_action.option_strings
                if (opts and opts[0] == arg) or group_action.dest == arg:
                    action._group_actions.remove(group_action)
                    return

    def handle_action(action):
        if isinstance(action, argparse._SubParsersAction):
            subparsers_actions = action.choices.values()
            for subparser in subparsers_actions:
                subparser: argparse.ArgumentParser
                remove_argument(subparser, "help")
                fmt = subparser.format_help().replace("usage: ", "# ")
                if len(fmt.strip().split("\n")) != 1:
                    print(fmt)
                for action in subparser._actions:
                    handle_action(action)

    print("\n")
    for action in parser._actions:
        handle_action(action)

Pandorapandour

Answer 9

A variation based entirely on @grundic answer.

Here, the parser handler is modified in subclass definition. So, no need to redefine '-h' in the code. The only change you need in the caller code is using subclass instead of stock ArgumentParser.

class VerboseArgumentParser(argparse.ArgumentParser):

    def format_help(self):

        help_messages = [super().format_help()]

        subparsers_actions = [
            action
            for action in self._actions
            if isinstance(action, argparse._SubParsersAction)
        ]

        for action in subparsers_actions:
            for choice, subparser in action.choices.items():
                help_messages.append(subparser.format_help())

        return '\n\n'.join(help_messages)


parser = VerboseArgumentParser(...)
# The rest of the code is intact.

You can also play with redefining subparser help messages to improve this solution even more

Here's an example of resulting output:

usage: program.py [-h] {sub1,sub2} ...

positional arguments:
  {sub1, sub2}          Subcommand choice help.
    sub1                Subcommand 1 help.
    sub2                Subcommand 2 help.

optional arguments:
  -h, --help            show this help message and exit


usage: program.py sub1 [-h] [--opt1]

optional arguments:
  -h, --help   show this help message and exit
  --opt1 OPT1  Option1 help message.


usage: program.py sub2 [-h] [--opt2 OPT2] [--opt3 OPT3]

optional arguments:
  -h, --help   show this help message and exit
  --opt2 OPT2  Option2 help message.
  --opt3 OPT3  Option3 help message.

Whyalla

Answer 10

This is a comprehensive recursive help system, complete with a utility class. The formatter can be easily replaced:

import argparse
from argparse import _SubParsersAction, _ArgumentGroup
from contextlib import suppress
from typing import TYPE_CHECKING, NamedTuple, Optional, Generator

class ParserInfo(NamedTuple):
    name: str
    parser: argparse.ArgumentParser
    level: int
    cmds: list[str]
    choice: argparse.Action
    has_subcommands: bool


# noinspection PyProtectedMember
class ParserUtil:
    """Exposes tools for manipulating and walking ArgumentParser objects."""

    # pylint: disable=protected-access

    @staticmethod
    def remove_subparser_action_group(parser):
        ag = ParserUtil._get_subparser_action_group(parser)
        if ag:
            parser._action_groups.remove(ag)

    @staticmethod
    def get_subparser(parser) -> _SubParsersAction:
        for action in parser._actions:
            if isinstance(action, _SubParsersAction):
                return action
        return None

    @staticmethod
    def remove_action(parser, tag):
        remove = None
        for action in parser._actions:
            if tag in action.option_strings:
                remove = action
                break

        if remove:
            parser._actions.remove(remove)
            for group in parser._action_groups:
                with suppress(ValueError):
                    group._group_actions.remove(remove)

    @staticmethod
    def _get_subparser_action_group(parser) -> Optional[_ArgumentGroup]:
        for ag in parser._action_groups:
            if not ag._group_actions:
                continue
            if isinstance(ag._group_actions[0], _SubParsersAction):
                return ag
        return None

    @staticmethod
    def _get_subcommands(parser):
        subs = ParserUtil.get_subparser(parser)
        for name, ent in subs._name_parser_map.items():
            choice = None
            for choice in subs._choices_actions:
                if name == choice.dest:
                    break
            yield name, ent, choice

    @staticmethod
    def walk_parsers(parser, name="", level=0, cmds=None) -> Generator[ParserInfo, None, None]:
        """Walks all nested subparsers.

        Yields a ParserInfo object containing:
            name: Command name
            parser: Parser object
            level: Depth level
            cmds: List of commands
            choice: Choice object
            has_subcommands: bool
        """
        if cmds is None:
            cmds = []

        if level == 0:
            # root parser
            yield ParserInfo(name, parser, level, [], argparse.Action("", ""), True)

        seen = set()
        for sub_name, subc, choice in ParserUtil._get_subcommands(parser):
            if subc in seen:
                continue
            seen.add(subc)
            cmd_ext = cmds + [sub_name]

            sub_ag = ParserUtil._get_subparser_action_group(subc)

            yield ParserInfo(sub_name, subc, level, cmd_ext, choice, bool(sub_ag))

            if sub_ag:
                yield from ParserUtil.walk_parsers(subc, sub_name, level + 1, cmd_ext)

def format_help_output(info: ParserInfo):
    if info.choice.help == argparse.SUPPRESS:
        return

    parser = info.parser
    # remove the "--help" action from printing on every subcommand
    ParserUtil.remove_action(parser, "--help")

    # suppress the "usage:" line when formatting help
    parser.usage = argparse.SUPPRESS

    if info.name:
        if not info.has_subcommands:
            full_cmd = " ".join(info.cmds[:-1] + [info.name])
            print(full_cmd, ":", info.choice.help)
        else:
            if info.level == 0:
                # top level commands get major sections
                print("==", info.name, ":", info.choice.help)
                print("\n")

    # remove the subparser action group auto help
    ParserUtil.remove_subparser_action_group(parser)

    # print top-level actions/info if available
    argp_help = parser.format_help()
    if argp_help.strip():
        print(textwrap.indent(argp_help, info.level * 4 * " "))

def show_help_for_parser(parser):
    parser.epilog = "all subcommands:"

    for info in ParserUtil.walk_parsers(parser):
        format_help_output(info)

Pandorapandour