Python argparse - Add argument to multiple subparsers

Question

My script defines one main parser and multiple subparsers. I want to apply the -p argument to some subparsers. So far the code looks like this:

parser = argparse.ArgumentParser(prog="myProg")
subparsers = parser.add_subparsers(title="actions")

parser.add_argument("-v", "--verbose",
                    action="store_true",
                    dest="VERBOSE",
                    help="run in verbose mode")

parser_create = subparsers.add_parser ("create", 
                                        help = "create the orbix environment")
parser_create.add_argument ("-p", 
                            type = int, 
                            required = True, 
                            help = "set db parameter")

# Update
parser_update = subparsers.add_parser ("update", 
                                        help = "update the orbix environment")
parser_update.add_argument ("-p", 
                            type = int, 
                            required = True, 
                            help = "set db parameter")

As you can see the add_arument ("-p") is repeated twice. I actually have a lot more subparsers. Is there a way to loop through the existing subparsers in order to avoid repetition?

For the record, I am using Python 2.7

Coquet

Related: #14919304 – Starveling

Also see example in documentation: parents – Tivoli

Answer 1

Update by @hpaulj

Due to changes in handling subparsers since 2011, it is a bad idea to use the main parser as a parent. More generally, don't try to define the same argument (same dest) in both main and sub parsers. The subparser values will overwrite anything set by the main (even the subparser default does this). Create separate parser(s) to use as parents. And as shown in the documentation, parents should use add_help=False.

---

Original answer

This can be achieved by defining a parent parser containing the common option(s):

import argparse

parent_parser = argparse.ArgumentParser(description="The parent parser")
parent_parser.add_argument("-p", type=int, required=True,
                           help="set db parameter")
subparsers = parent_parser.add_subparsers(title="actions")
parser_create = subparsers.add_parser("create", parents=[parent_parser],
                                      add_help=False,
                                      description="The create parser",
                                      help="create the orbix environment")
parser_create.add_argument("--name", help="name of the environment")
parser_update = subparsers.add_parser("update", parents=[parent_parser],
                                      add_help=False,
                                      description="The update parser",
                                      help="update the orbix environment")

This produces help messages of the format:

parent_parser.print_help()

Output:

usage: main.py [-h] -p P {create,update} ...
The parent parser
optional arguments:
  -h, --help       show this help message and exit
  -p P             set db parameter
actions:
  {create,update}
    create         create the orbix environment
    update         update the orbix environment

parser_create.print_help()

Output:

usage: main.py create [-h] -p P [--name NAME] {create,update} ...
The create parser
optional arguments:
  -h, --help       show this help message and exit
  -p P             set db parameter
  --name NAME      name of the environment
actions:
  {create,update}
    create         create the orbix environment
    update         update the orbix environment

However, if you run your program, you will not encounter an error if you do not specify an action (i.e. create or update). If you desire this behavior, modify your code as follows.

<...>
subparsers = parent_parser.add_subparsers(title="actions")
subparsers.required = True
subparsers.dest = 'command'
<...>

This fix was brought up in this SO question which refers to an issue tracking a pull request.

Thermel

From the docs: "when a help message is requested from a subparser, only the help for that particular parser will be printed. The help message will not include parent parser or sibling parser messages." That seems like a major shortcoming of this strategy. – Divergent

I ended up adding a dummy argument to the root parser with a help message. – Divergent

@RyneEverett: that manual part is confusing and might be outdated, because it seems at least in Python 3.5.3 subparser --help includes arguments from the parent parser. – Viva

I've edited the answer showing --help outputs for parent and child parsers. Notice the help message for create shows both "create" and all actions, which is incorrect. The looping answers do not suffer from this issue – Wantage

The idea seems correct, however the example code is somehow weird, because in this case one would not need any parent parsers and could add parameters directly to main parser – Dulaney

@TheGodfather Yes this is true, however you lose the hierarchy in doing so. There will be cases where you definitely do not want to mislead your user that e.g. the arguments for create can not be provided with the update command. – Wantage

@Wantage no-no, I mean that usually you still have other arguments and other subparsers and other root-level commands (see in my answer-example). Of course you still use parent, but only for the stuff it is actually a parent, not for every argument. In the given example -p should be argument only for two given subparsers, but not for the whole script (think of having 5 other subparsers which do not have this arg). – Dulaney

@TheGodfather I see what you are saying now. See my reply to your answer (shortly...). I will update this answer to reflect both options – Wantage

I think the reason it behaves oddly is that a "parent parser" is not actually the same thing as a "parser with subparsers". (note that the docs never show the two features used together) Pretty sure you need a top-level parser, a separate "parent" parser with the global args on it, and subparsers on the former with their parents= set to the latter. – Arborvitae

Could you please check my question #59391584 ? Probably you know the answer. Thanks. – Endive

I'm running into an issue where I cannot get access to the help messages for a subcommand's options. I have a main parser, and a separate parent parser with add_help=False which holds arguments which are common to both subcommands which is added via add_parser(..., parents=[parent_parser]. When I then attempt to use command subcommand -h to view the help for those common args I get an unrecognized arguments: -h error. – Statolith

@Statolith If your setup is global_p (with all global args), toplevel_p (with parent global_p) and then subparsers (created from toplevel_p), and then parsers (added to subparsers with parent global_p), then the parser that requires add_help=False is actually JUST global_p, not toplevel_p. NOTE: if you do not own global_p or did not create/init global_p with add_help=False you can manually remove it by searching for the help action in global_p._actions and calling global_p._remove_action() on it. – Joellenjoelly

Answer 2

The accepted answer is correct; the proper way is to use parent parsers. However, the example code IMO was not really solving the problem. Let me add my few cents to provide a more suitable example.

The main difference with accepted answer is the explicit possibility to have some root-level arguments (like --verbose) as well as shared arguments only for some subparsers (-p only for the create and update subparsers but not for others)

# Same main parser as usual
parser = argparse.ArgumentParser()

# Usual arguments which are applicable for the whole script / top-level args
parser.add_argument('--verbose', help='Common top-level parameter',
                    action='store_true', required=False)

# Same subparsers as usual
subparsers = parser.add_subparsers(help='Desired action to perform', dest='action')

# Usual subparsers not using common options
parser_other = subparsers.add_parser("extra-action", help='Do something without db')

# Create parent subparser. Note `add_help=False` and creation via `argparse.`
parent_parser = argparse.ArgumentParser(add_help=False)
parent_parser.add_argument('-p', help='add db parameter', required=True)

# Subparsers based on parent

parser_create = subparsers.add_parser("create", parents=[parent_parser],
                                      help='Create something')
# Add some arguments exclusively for parser_create

parser_update = subparsers.add_parser("update", parents=[parent_parser],
                                      help='Update something')
# Add some arguments exclusively for parser_update 

This is the top-level help message (note that -p parameter is not shown here, which is exactly what you would expect—because it is specific to some subparsers):

>>> parser.print_help()
usage: [-h] [--verbose] {extra-action,create,update} ...

positional arguments:
  {extra-action,create,update}
                        Desired action to perform
    extra-action        Do something without db
    create              Create something
    update              Update something

optional arguments:
  -h, --help            show this help message and exit
  --verbose             Common top-level parameter

And the help message for the create action:

>>> parser_create.print_help()
usage:  create [-h] -p P

optional arguments:
  -h, --help  show this help message and exit
  -p P        add db parameter

Dulaney

So, in my previous edit to the main answer, I include the "-p" option in the parent parser to show how users may have common options between all subparsers. I then showed that a "--name" option could be added to a single subparser that is unique to the create command/action. The main difference in our solutions is in how help is shown. Here is my output and here is your output. – Wantage

@Wantage you do not need parent parser to add arguments for all (this can be done just with normal parser) ;) Different help is just an illustration for much bigger difference here. BTW I also added help output to my answer. – Dulaney

I completely agree, it is not needed but some may want to use a mix of both. Your help messages are definitely cleaner, thanks for adding! – Wantage

The separate parent parser is the only way to share positional arguments between multiple subcommands. – Teakettle

This is a brilliantly clear example, this answer needs more upvotes folks. – Pneumectomy

Should this work in this way: --verbose create -p 2 update -p 3. For me it says: "usage: argparser.py [-h] [--verbose] {extra-action,create,update} ... argparser.py: error: unrecognized arguments: update" – Endive

@Endive no, subparser's actions are mutually exclusive. First you run --verbose create -p 2 and then you run --verbose update -p 3 – Dulaney

how come parent_parses is included in arguments if parent_parser.parse_args() was never done? – Furnish

@Furnish I don't really get what you mean, print_help just prints help and doesn't require calling parse_args. But in the product you'd use parser.parse_args() (or parent_parser) – Dulaney

@TheGodfather yes, that's right, what I mean is if we do parser.parse_args(), then how come we can also use '-p' argument from parent_parser without ever doing parent_parser.parse_args()? – Furnish

I am trying to avoid using another library but let me get if I understand this. We have to create separate parser to use it as parent in subparser of the original parser. That's the only way to get the proper print help. And we still can't use --verbose at the end. It has to be at the beginning? – Furnish

Answer 3

You can also loop over the subparsers and add the same option to all of them.

parser = argparse.ArgumentParser(prog="myProg")
subparsers = parser.add_subparsers(title="actions")
parser.add_argument("-v", "--verbose",
                    action="store_true",
                    dest="VERBOSE",
                    help="run in verbose mode")

parser_create = subparsers.add_parser ("create", 
                                        help = "create the orbix environment")
parser_update = subparsers.add_parser ("update", 
                                        help = "update the orbix environment")

for subparser in [parser_create, parser_update]:
    subparser.add_argument ("-p", 
                            type = int, 
                            required = True, 
                            help = "set db parameter")

Amorette

i think that's a bad aproach, cause you need to loop over parameters. it's more DRY and redable the Sven Marnach answer. – Parish

Actually this is MUCH simpler and more readable than the more upvoted solutions. For loops do NOT violate DRY principle. – Effeminate

Answer 4

You can loop over your subparsers in the following way.

for name, subp in subparsers.choices.items():
    print(subp)
    subp.add_argument(dest='g', help='Input for g variable', default=7, type=int)

Note that by using subparsers.choices that you avoid needing to hard-code all of the subparsers.

Outfitter

This is the same as @JanK's answer – Wantage

@Wantage Not quite. It is similar, i'll give you that. The difference lies in the iterable. JanK's answer uses a list of parsers, mine uses subparsers.choices.items(). The advantage of subparsers.choices.items()is that subparsers.choices.items() always contains all used subparsers. When using a list you will have to change it when subparsers are added or removed. I do have to give Jank's credit for the idea, i just took it a step further because i dont like to hardcode and maintain the list. – Outfitter

@Gerald Kool Ah, I see. I missed that when looking this answer over! Thank you – Wantage

This works well for 1 level parsers, but recursively grabbing subparsers can be tough if you have a deeply nested command tree. I created some helper functions to make this easier here: github.com/turtlemonvh/argparse-subparser-filter – Chirurgeon

I like this one because I can write --verbose at the end, not beginning – Furnish

@Chirurgeon you say to write p = generate_parsers(), but where is that function in your code? – Furnish

generate_parser isn't a function in my code. It's just an example of a function you would write that returns a argparse.ArgumentParser object. You don't need a function at all - you just need a argparse.ArgumentParser that you can use as an argument to the get_parsers function in my code. – Chirurgeon

FYI this will fail if you have an alias added to any of the subparsers. In that case, subparsers.choices.items() will include the same subparser twice and raise an error for the second one: argparse.ArgumentError: argument -f/--foo: conflicting option strings: -f, --foo – Vector