Group Django commands to a folder inside the same app
Asked Answered
L

4

6

Is it allowed to group custom Django commands to separate folders inside the same Django app?

I have a lot of them and wanted to group them logically by purpose. Created folders but Django can't find them.

Maybe I'm trying to run them wrong. Tried:

  • python manage.py process_A_related_data
  • the same plus imported all commands in __init__.py
  • python manage.py folderA process_A_related_data
  • python manage.py folderA.process_A_related_data
  • python manage.py folderA/process_A_related_data

Got following error:

Unknown command: 'folderA/process_A_related_data'
Type 'manage.py help' for usage.
Lacylad answered 9/12, 2019 at 10:38 Comment(2)
Sorry but I don't get it. is answer to your question how to have django commands in different folders rather than a management/commands?Sizeable
@Sizeable I want to group them inside management/commands. Like for example: management/commands/folderA, management/commands/folderB, etc. When I move them to these folders and import in __init__.py - I see exception shown above.Lacylad
C
9

I think you can create a basic custom command which will run other commands from relevent folders. Here is an approach you can take:

First make a folder structure like this:

management/
    commands/
       folder_a/
          process_A_related_data.py
       folder_b/
          process_A_related_data.py
       process_data.py

Then inside process_data.py, update the command like this:

from django.core import management
from django.core.management.base import BaseCommand

import importlib

class Command(BaseCommand):
   help = 'Folder Process Commands'

   def add_arguments(self, parser):
       parser.add_argument('-u', '--use', type=str, nargs='?', default='folder_a.process_A_related_data')
    
   def handle(self, *args, **options):
       try:
           folder_file_module = options['use'] if options['use'].startswith('.') else '.' + options['use']
           command = importlib.import_module(folder_file_module, package='your_app.management.commands')
           management.call_command(command.Command())
       except ModuleNotFoundError:
           self.stderr.write(f"No relevent folder found: {e.name}")

Here I am using call_command method to call other managment commands.

Then run commands like this:

python manage.py process_data --use folder_a.process_A_related_data

Finally, if you want to run commands like python manage.py folder_a.process_A_related_data, then probably you need to change in manage.py. Like this:

import re
...

try:
    from django.core.management import execute_from_command_line
except ImportError as exc:
    raise ImportError(
        "Couldn't import Django. Are you sure it's installed and "
        "available on your PYTHONPATH environment variable? Did you "
        "forget to activate a virtual environment?"
    ) from exc

if re.search('folder_[a-z].*', sys.argv[-1]):
    new_arguments = sys.argv[:-1] + ['process_data','--use', sys.argv[-1]]
    execute_from_command_line(new_arguments)
else:
    execute_from_command_line(sys.argv)
Chirpy answered 10/3, 2020 at 7:4 Comment(1)
Will test later to check, but looks like this should achieve the goal.Lacylad
H
1

You should be able to partition the code by using mixins (I have not tried this in this context, though)

A standard management command looks like

from django.core.management.base import BaseCommand

class Command(BaseCommand):
   help = 'FIXME A helpful comment goes here'

    def add_arguments(self, parser):
        parser.add_argument( 'name', ...)
        # more argument definitions

    def handle(self, *args, **options):  
        # do stuff

Which can probably be replaced by a "stub" in app/management/commands:

from wherever.commands import FooCommandMixin
from django.core.management.base import BaseCommand    
class Command(FooCommandMixin, BaseCommand): 
    # autogenerated -- do not put any code in here!
    pass

and in wherever/commands

class FooCommandMixin( object):
   help = 'FIXME A helpful comment goes here'

    def add_arguments(self, parser):
        parser.add_argument( 'name', ...)
        # more argument definitions

    def handle(self, *args, **options):  
        # do the work

It would not be hard to write a script to go through a list of file names or paths (using glob.glob) using re.findall to identify appropriate class declarations, and to (re)generate a matching stub for each in the app's management/commands folder.

Also/instead Python's argparse allows for the definition of sub-commands. So you should be able to define a command that works like

./manage.py foo bar --aa --bb something --cc  and
./manage.py foo baz --bazzy a b c 

where the syntax after foo is determined by the next word (bar or baz or ...). Again I have no experience of using subcommands in this context.

Hyaluronidase answered 4/3, 2020 at 10:15 Comment(2)
As I understood, you propose to create command which will be a "runner" for my subcommands using first argument as name for command, correct?Lacylad
Yes, argparse subcommands ought to let you have one management command with the next word determining what to do and what syntax applies. You could then dispatch in the "top" handle method to functions or mixins whose code is located elsewhere, and these do the work. I've never tried subcomands in this context, though.Hyaluronidase
C
0

I found no mention of support for this feature in the release notes. It looks to be that this is still not supported as of version Django 3.0. I would suggest that you use meaningful names for your files that help you specify. You could always come up w/ a naming convention!

Celestinacelestine answered 3/3, 2020 at 14:34 Comment(4)
Meaningful naming convention will not help, as it is still not convenient to work with big list of files in a single folder in IDE.Lacylad
@bitzbytes why from version 3.0? What was changed? Did you mean version of Django or Python?Lacylad
@Lacylad as of version 3.0 of Django. They still have no support for the feature you are requesting. You can see a similar question asked here: https://mcmap.net/q/837414/-django-split-management-commands-into-subfolders I guess if you really needed to, you could take on the advice given here.Celestinacelestine
Thanks. Looks like I'm looking for patch like described there.Lacylad
F
0

A workaround could be: create a specific Django "satellite" app for each group of management commands.

In recent version of Django, the requirements for a Python module to be an app are minimal: you won't need to provide any fake models.py or other specific files as happened in the old days.

While far from perfect from a stylistic point of view, you still gain a few advantages:

  • no need to hack the framework at all
  • python manage.py will list the commands grouped by app
  • you can control the grouping by providing suitable names to the apps
  • you can use these satellite apps as container for specific unit tests

I always try to avoid fighting against the framework, even when this means to compromise, and sometimes accept it's occasional design limitations.

Freewill answered 3/3, 2020 at 15:15 Comment(6)
In that case, I'll get a lot of redundant apps. About "always try to avoid fighting against the framework" - you are totally right. That's why I'm looking for right tool for that.Lacylad
True ! I understand your suggestion, and agree on it, but ... You could surely succeed in partitioning the management commands in subfolders with some code twist, but please ask to yourself: will other "standard" management command still work with the new project layout ? For example: will makemessages work ? Maybe yes, maybe no ;) You might face some headaches much later in the project developmentFreewill
But makemessages are running by django-admin not manage.py ;)Lacylad
actually I'm looking for solution that will work and not break anything.Lacylad
You can run any management command with manage.py; django-admin just provides a subset ... and notably startproject which is needed at the very beginning to generate the manage.py script ;) All in all, makemessages was just an example of a command which needs to navigate the project subfolders, and as such could get confused by an unexpected layoutFreewill
"actually I'm looking for solution that will work and not break anything" sure, we always do ! and we also hope that it won't break anything with future versions of the frameworkFreewill

© 2022 - 2024 — McMap. All rights reserved.