How should I implement "nested" subcommands in Python?

Question

Implementing "nested" subcommands in Python with cmdln.

I'm not sure I'm using the right terminology here. I'm trying to implement a commandline tool using cmdln that allows for "nested" subcommands. Here is a real world example:

git svn rebase

What is the best way of implementing this? I've been searching for more information on this in the doc, here and the web at large, but have come up empty. (Perhaps I was searching with the wrong terms.)

Short of an undocumented feature that does this automatically, my initial thought was to have the previous subcommand handler determine that there is another subcommand and dispatch the command dispatcher again. I've looked at the internals of cmdln though and the dispatcher is a private method, _dispatch_cmd. My next thought is to create my own sub-sub-command dispatcher, but that seems less than ideal and messy.

Any help would be appreciated.

Daisie

Answer 1

argparse makes sub-commands very easy.

Disharmony

The company I am working for has a v2.6 baseline so using argparse is an issue in that it would have to be included as an external library and only loaded if needed. Far from impossible, just not ideal. As for the cmdln library, gives me a nice bit of baseline functionality that I'd prefer not to recreate. That said I am opposed to using something else. – Daisie

Not for more than 2 levels of nesting (main parser + one layer of subparsers). Arbitrary nesting does not work. It results in unexpected behavior and parse failures. – Shovel

Answer 2

Late to the party here, but I've had to do this quite a bit and have found argparse pretty clunky to do this with. This motivated me to write an extension to argparse called arghandler, which has explicit support for this - making is possible implement subcommands with basically zero lines of code.

Here's an example:

from arghandler import *

@subcmd
def push(context,args):
    print 'command: push'

@subcmd
def pull(context,args):
    print 'command: pull'

# run the command - which will gather up all the subcommands
handler = ArgumentHandler()
handler.run()

Gaylagayle

Does arghandler support arbitrary nesting of subcommands? I spent 2 hours and cannot get it to work for three levels. OP was explicitly giving an example for three levels (git svn rebase). I find the readme of your arghandler repo quite frustrating in this regard. More than 2 levels of nesting is probably the main reason people look for an alternative to argparse and might come across your repo that way, but it does not give an example for > 2 levels. Argparse can do 2 levels as well, so why introduce another dependency for that? Just a tip for the readme. – Shovel

Answer 3

I feel like there's a slight limitation with sub_parsers in argparse, if say, you have a suite of tools that might have similar options that might spread across different levels. It might be rare to have this situation, but if you're writing pluggable / modular code, it could happen.

I have the following example. It is far-fetched and not well explained at the moment because it is quite late, but here it goes:

Usage: tool [-y] {a, b}
  a [-x] {create, delete}
    create [-x]
    delete [-y]
  b [-y] {push, pull}
    push [-x]
    pull [-x]

from argparse import ArgumentParser

parser = ArgumentParser()
parser.add_argument('-x', action = 'store_true')
parser.add_argument('-y', action = 'store_true')

subparsers = parser.add_subparsers(dest = 'command')

parser_a = subparsers.add_parser('a')
parser_a.add_argument('-x', action = 'store_true')
subparsers_a = parser_a.add_subparsers(dest = 'sub_command')
parser_a_create = subparsers_a.add_parser('create')
parser_a_create.add_argument('-x', action = 'store_true')
parser_a_delete = subparsers_a.add_parser('delete')
parser_a_delete.add_argument('-y', action = 'store_true')

parser_b = subparsers.add_parser('b')
parser_b.add_argument('-y', action = 'store_true')
subparsers_b = parser_b.add_subparsers(dest = 'sub_command')
parser_b_create = subparsers_b.add_parser('push')
parser_b_create.add_argument('-x', action = 'store_true')
parser_b_delete = subparsers_b.add_parser('pull')
parser_b_delete.add_argument('-y', action = 'store_true')

print parser.parse_args(['-x', 'a', 'create'])
print parser.parse_args(['a', 'create', '-x'])
print parser.parse_args(['b', '-y', 'pull', '-y'])
print parser.parse_args(['-x', 'b', '-y', 'push', '-x'])

Output

Namespace(command='a', sub_command='create', x=True, y=False)
Namespace(command='a', sub_command='create', x=True, y=False)
Namespace(command='b', sub_command='pull', x=False, y=True)
Namespace(command='b', sub_command='push', x=True, y=True)

As you can see, it is hard to distinguish where along the chain each argument was set. You could solve this by changing the name for each variable. For example, you could set 'dest' to 'x', 'a_x', 'a_create_x', 'b_push_x', etc., but that would be painful and hard to separate out.

An alternative would be to have the ArgumentParser stop once it reaches a subcommand and pass the remaining arguments off to another, independent parser, so it could generates separate objects. You can try to achieve that by using 'parse_known_args()' and not defining arguments for each subcommand. However, that would not be good because any un-parsed arguments from before would still be there and might confuse the program.

I feel a slightly cheap, but useful workaround is to have argparse interpret the following arguments as strings in a list. This can be done by setting the prefix to a null-terminator '\0' (or some other 'hard-to-use' character) - if the prefix is empty, the code will throw an error, at least in Python 2.7.3.

Example:

parser = ArgumentParser()
parser.add_argument('-x', action = 'store_true')
parser.add_argument('-y', action = 'store_true')
subparsers = parser.add_subparsers(dest = 'command')
parser_a = subparsers.add_parser('a' prefix_chars = '\0')
parser_a.add_argument('args', type = str, nargs = '*')

print parser.parse_args(['-xy', 'a', '-y', '12'])

Output:

Namespace(args=['-y', '12'], command='a', x=True, y=True)

Note that it does not consume the second -y option. You can then pass the result 'args' to another ArgumentParser.

Drawbacks:

• Help might not be handled well. Would have to make some more workaround with this
• Encountering errors might be hard to trace and require some additional effort to make sure error messages are properly chained.
• A little bit more overhead associated with the multiple ArgumentParsers.

If anybody has more input on this, please let me know.

Copyread

Answer 4

Update for year 2020!
Click library has easier usage
Fire is a cool library for making your app command line featured!

Mortonmortuary

This should be the accepted answer. Click makes arbitrarily nested subparsers possible, and easy at that! – Shovel