How to Set a Default Subparser using Argparse Module with Python 2.7

Question

I'm using Python 2.7 and I'm trying to accomplish a shell like behavior using argparse. My issue, in general, that I cannot seem to find a way, in Python 2.7, to use argparse's subparsers as optional. It's kind of hard to explain my issue so I'll describe what I require from my program.

The program has 2 modes of work:

1. Starting the program with a given command (each command has it's own additional arguments) and additional arguments will run a specific task.
2. Starting the program without a command will start a shell-like program that can take a line of arguments and process them as if the program was called with the given line as it's arguments.

So, if for example my program supports 'cmd1' and 'cmd2' commands, I could use it like so:

• python program.py cmd1 additional_args1
• python program.py cmd2 additional_args2

or with shell mode:

• python program.py
  • cmd1 additional_args1
  • cmd2 additional_args2
  • quit

In addition, I also want my program to be able to take optional global arguments that will effect all commands.

For that I'm using argparse like so (This is a pure example):

parser = argparse.ArgumentParser(description="{} - Version {}".format(PROGRAM_NAME, PROGRAM_VERSION))

parser.add_argument("-i", "--info",  help="Display more information")

subparsers = parser.add_subparsers()

parserCmd1 = subparsers.add_parser("cmd1", help="First Command")
parserCmd1.set_defaults(func=cmd1)

parserCmd2 = subparsers.add_parser("cmd2", help="Second Command")
parserCmd2.add_argument("-o", "--output", help="Redirect Output")
parserCmd2.set_defaults(func=cmd2)

So I can call cmd1 (with no additional args) or cmd2 (with or without -o flag). And for both I can add flag -i to display even more information of the called command.

My issue is that I cannot activate shell mode, because I have to provide cmd1 or cmd2 as an argument (because of using subparsers which are mandatory)

Restrictions:

• I cannot use Python 3 (I know it can be easily done there)
• Because of global optional arguments I cannot check to see if I get no arguments to skip arg parsing.
• I don't want to add a new command to call shell, it must be when providing no command at all

So how can I achieve This kind of behavior with argparse and python 2.7?

Grummet

Just a quick note - the fact that subparsers are optional is a bug. They used to be required (as a normal positional is), but in an unrelated change some years ago, subparsers fell through the cracks. I"ll have to study your question in more detail to understand why you think there's a difference between Py2 and Py3 in this regard. – Priam

Answer 1

Another idea is to use a 2 stage parsing. One handles 'globals', returning strings it can't handle. Then conditionally handle the extras with subparsers.

import argparse

def cmd1(args):
    print('cmd1', args)
def cmd2(args):
    print('cmd2', args)

parser1 = argparse.ArgumentParser()

parser1.add_argument("-i", "--info",  help="Display more information")

parser2 = argparse.ArgumentParser()
subparsers = parser2.add_subparsers(dest='cmd')

parserCmd1 = subparsers.add_parser("cmd1", help="First Command")
parserCmd1.set_defaults(func=cmd1)

parserCmd2 = subparsers.add_parser("cmd2", help="Second Command")
parserCmd2.add_argument("-o", "--output", help="Redirect Output")
parserCmd2.set_defaults(func=cmd2)

args, extras = parser1.parse_known_args()
if len(extras)>0 and extras[0] in ['cmd1','cmd2']:
    args = parser2.parse_args(extras, namespace=args)
    args.func(args)
else:
    print('doing system with', args, extras)

sample runs:

0901:~/mypy$ python stack46667843.py -i info
('doing system with', Namespace(info='info'), [])
0901:~/mypy$ python stack46667843.py -i info extras for sys
('doing system with', Namespace(info='info'), ['extras', 'for', 'sys'])
0901:~/mypy$ python stack46667843.py -i info cmd1
('cmd1', Namespace(cmd='cmd1', func=<function cmd1 at 0xb74b025c>, info='info'))
0901:~/mypy$ python stack46667843.py -i info cmd2 -o out
('cmd2', Namespace(cmd='cmd2', func=<function cmd2 at 0xb719ebc4>, info='info', output='out'))
0901:~/mypy$ 

Priam

Thanks for the reply, I was able to take your answer and implement what I needed. – Grummet

The downside I am seeing to this solution is that the top level parser help output no longer documents the sub-commands – Punkah

@jdi, parser1 should include add_help=False so it doesn't respond to the help request. – Priam

@Priam yea I had to disable the automatic help on the top parser and then manually implement the help flags. Then I would need to check the help flag on the top parser only after the subcommand parser delegation has been checked first. – Punkah

Answer 2

A bug/issue (with links) on the topic of 'optional' subparsers.

https://bugs.python.org/issue29298

Notice that this has a recent pull request.

---

With your script and the addition of

args = parser.parse_args()
print(args)

results are

1008:~/mypy$ python3 stack46667843.py 
Namespace(info=None)
1009:~/mypy$ python2 stack46667843.py 
usage: stack46667843.py [-h] [-i INFO] {cmd1,cmd2} ...
stack46667843.py: error: too few arguments
1009:~/mypy$ python2 stack46667843.py cmd1
Namespace(func=<function cmd1 at 0xb748825c>, info=None)
1011:~/mypy$ python3 stack46667843.py cmd1
Namespace(func=<function cmd1 at 0xb7134dac>, info=None)

I thought the 'optional' subparsers affected both Py2 and 3 versions, but apparently it doesn't. I'll have to look at the code to verify why.

---

In both languages, subparsers.required is False. If I set it to true

subparsers.required=True

(and add a dest to the subparsers definition), the PY3 error message is

1031:~/mypy$ python3 stack46667843.py
usage: stack46667843.py [-h] [-i INFO] {cmd1,cmd2} ...
stack46667843.py: error: the following arguments are required: cmd

So there's a difference in how the 2 versions test for required arguments. Py3 pays attention to the required attribute; Py2 (apparently) uses the earlier method of checking whether the positionals list is empty or not.

---

Checking for required arguments occurs near the end of parser._parse_known_args.

Python2.7 includes

    # if we didn't use all the Positional objects, there were too few
    # arg strings supplied.
    if positionals:
        self.error(_('too few arguments'))

before the iteration that checks action.required. That's what's catching the missing cmd and saying too few arguments.

So a kludge is to edit your argparse.py and remove that block so it matches the corresponding section of the Py3 version.

Priam

Hi, thank you for the response. I also cannot edit argparse, for the same reason I cannot use Python3. This is going to be a tool used by people in the company I'm working at. The only guarantee I have is that they have Python 2.7 installed. – Grummet