Is it possible to create subparsers in a django management command?
Asked Answered
M

2

16

Title really says it all, but I currently have this, but it doesn't work:

class Command(BaseCommand):
    help = ("Functions related to downloading, parsing, and indexing the  "
            "content")

    def add_arguments(self, parser):
        subparsers = parser.add_subparsers()

        download_parser = subparsers.add_parser(
            'download',
            help='Using a local CSV, download the XML data for content. '
                 'Output is sent to the log.'
        )
        download_parser.add_argument(
            '--start_line',
            type=int,
            default=0,
            help='The line in the file where you wish to start processing.'
        )

        # Add an argparse parser for parsing the content. Yes, this is
        # a bit confusing.
        content_parser_parser = subparsers.add_parser(
            'parse',
            help="Look at the file system and parse everything you see so that "
                 "we have content in the databse."
        )
        content_parser_parser.add_argument(
            '--start_item',
            type=int,
            default=0,
            help="Assuming the content is sorted by file name, this item is "
                 "the one to start on."
        )

My specific idea is to create one command that has subcommands for downloading XML content or for parsing it into the database.

Mcfarland answered 18/4, 2016 at 23:54 Comment(2)
Without knowing what is in the parser already, or what django does it with it later, I can't say. Off hand your subparser definitions look fine. But as you may see from other SO questions, making subparsers work with other arguments, positional and/or optionals, can be tricky. Just as a diagnostic, add print parser._actions at the start of your function.Circularize
#31919601, is one previous question involving both argparse and django. Looks like django used to use optparse, but has recently added the argparse alternative.Circularize
J
23

Django 2.1 and above

In Django 2.1 and above, adding a subcommand is trivial:

from django.core.management.base import BaseCommand

class Command(BaseCommand):

    def add_arguments(self, parser):
        subparsers = parser.add_subparsers(title="subcommands",
                                           dest="subcommand",
                                           required=True)

Then you use subparser the same way you'd do if you were writing a non-Django application that uses argparse. For instance, if you want a subcommand named foo that may take the --bar argument:

foo = subparsers.add_parser("foo")
foo.set_defaults(subcommand=fooVal)
foo.add_argument("--bar")

The value fooVal is whatever you decide the subcommand option should be set to when the user specifies the foo subcommand. I often set it to a callable.

Older versions of Django

It is possible but it requires a bit of work:

from django.core.management.base import BaseCommand, CommandParser

class Command(BaseCommand):

    [...]

    def add_arguments(self, parser):
        cmd = self

        class SubParser(CommandParser):

            def __init__(self, **kwargs):
                super(SubParser, self).__init__(cmd, **kwargs)

        subparsers = parser.add_subparsers(title="subcommands",
                                           dest="subcommand",
                                           required=True,
                                           parser_class=SubParser)

When you call add_subparsers by default argparse creates a new parser that is of the same class as the parser on which you called add_subparser. It so happens that the parser you get in parser is a CommandParser instance (defined in django.core.management.base). The CommandParser class requires a cmd argument before the **kwargs (whereas the default parser class provided by argparse only takes **kwargs):

def __init__(self, cmd, **kwargs):

So when you try to add the subparser, it fails because the constructor is called only with **kwargs and the cmd argument is missing.

The code above fixes the issue by passing in parser_class argument a class that adds the missing parameter.

Things to consider:

  1. In the code above, I create a new class because the name parser_class suggests that what should be passed there is a real class. However, this also works:

    def add_arguments(self, parser):
        cmd = self
        subparsers = parser.add_subparsers(
            title="subcommands",
            dest="subcommand",
            required=True,
            parser_class=lambda **kw: CommandParser(cmd, **kw))
    

    Right now I've not run into any issues but it is possible that a future change to argparse could make using a lambda rather than a real class fail. Since the argument is called parser_class and not something like parser_maker or parser_manufacture I would consider such a change to be fair game.

  2. Couldn't we just pass one of the stock argparse classes rather than pass a custom class in parser_class? There would be no immediate problem, but there would be unintended consequences. The comments in CommandParser show that the behavior of argparse's stick parser is undesirable for Django commands. In particular, the docstring for the class states:

    """
    Customized ArgumentParser class to improve some error messages and prevent
    SystemExit in several occasions, as SystemExit is unacceptable when a
    command is called programmatically.
    """
    

    This is a problem that Jerzyk's answer suffers from. The solution here avoids that problem by deriving from CommandParser and thus providing the correct behavior needed by Django.

Juarez answered 24/5, 2016 at 13:6 Comment(5)
Looks like a change has been made and this doesn't work any more with Django 2.1/Python3.6: parser_class=lambda **kw: CommandParser(self, **kw): TypeError: __init__() takes 1 positional argument but 2 were givenChockablock
@Chockablock My sites are still on 1.11. I looked at the relevant changes and it seems to me the changes made in 2.1 entirely eliminate the need for a workaround to pass the cmd argument. In other words, you should just be able to add a subparser by calling parser.add_subparsers without needing to use the parser_class arguments. Did you try it?Juarez
works great. I couldn't see the forest for the trees there :) subparsers.required=True seems to be needed as well, maybe it was by default previously?Chockablock
It seems that the need to explicitly set subparsers.required=True came about somewhere in the Python 3.x series, but did not affect the 2.7.x series. I'm still on 2.7 due to 3rd party libraries that have still not converted to 3.x so the issue does not affect my code. See this answer for an explanation and, for the gory details, the bug report that the answer refers to.Juarez
When there are errors in the subparsers arguments (number of args, string when number expected, etc), to prevent raising a CommandError and show instead a helpful message from argparse, add called_from_command_line=True to the arguments of add_parser().Daumier
D
3

you can add it and it was pretty simple:

class Command(BaseCommand):
    help = 'dump/restore/diff'

    def add_arguments(self, parser):
        parser.add_argument('-s', '--server', metavar='server', type=str, 
                            help='server address')
        parser.add_argument('-d', '--debug', help='Print lots of debugging') 

        subparsers = parser.add_subparsers(metavar='command',
                                           dest='command',
                                           help='sub-command help')
        subparsers.required = True

        parent_parser = argparse.ArgumentParser(add_help=False)
        parent_parser.add_argument('machine', metavar='device', type=str)
        parent_parser.add_argument('-e', '--errors', action='store_true')

        parser_dump = subparsers.add_parser('dump', parents=[parent_parser],
                                            cmd=self)
        parser_dump.add_argument('-i', '--indent', metavar='indent', type=int,                                   
                                  default=None, help='file indentation')

        parser_restore = subparsers.add_parser('restore',             
                                               parents=[parent_parser],
                                               cmd=self)
        parser_restore.add_argument('infile', nargs='?', 
                                    type=argparse.FileType('r'), 
                                    default=sys.stdin)

        parser_diff = subparsers.add_parser('diff', parents=[parent_parser], 
                                            cmd=self)
        parser_diff.add_argument('infile', nargs='?', 
                                 type=argparse.FileType('r'),
                                 default=sys.stdin)
Diestock answered 5/6, 2016 at 2:26 Comment(1)
You are using the vanilla argparse here that will do system exit. This is undesireable so that is why Django is inheriting the class and getting rid of that behavior. So this is not a recommended way to do it if you care about keeping to the same standards as the Django projectZepeda

© 2022 - 2024 — McMap. All rights reserved.