Run startup code in the parent with Django and Gunicorn

Question

I need my code run at Django application startup, before Django starts listening for incoming connections. Running my code upon the first HTTP request is not good enough. When I use Gunicorn, my code must run in the parent process, before it forks.

https://mcmap.net/q/49966/-where-to-put-django-startup-code doesn't seem to work in Django 1.4.2: it doesn't run the Middleware's __init__ method until the first request is received. Ditto for adding code to urls.py.

A quick Google search didn't reveal anything useful.

Americanize

Is there something in the Django site you need to run, or are you forking off daemon processes or something? – Predella

Forking daemons is dangerous - you can make nephilim that way :/ – Altagraciaaltaic

I want to run some quick checks before the site can go live. I won't spawn any daemons. – Americanize

Answer 1

This is old but in Gunicorn version 19.0 and above, you can create a custom script to run your application and include the startup code that you need there. Here is an example script using a django app:

#!/usr/bin/env python

"""
Script for running Gunicorn using our WSGI application
"""

import multiprocessing

from gunicorn.app.base import BaseApplication

from myapp.wsgi import application  # Must be imported first


class StandaloneApplication(BaseApplication):
    """Our Gunicorn application."""

    def __init__(self, app, options=None):
        self.options = options or {}
        self.application = app
        super().__init__()

    def load_config(self):
        config = {
            key: value for key, value in self.options.items()
            if key in self.cfg.settings and value is not None
        }
        for key, value in config.items():
            self.cfg.set(key.lower(), value)

    def load(self):
        return self.application


if __name__ == '__main__':
    gunicorn_options = {
        'bind': '0.0.0.0:8080',
        'workers': (multiprocessing.cpu_count() * 2) + 1,
    }

    # Your startup code here

    StandaloneApplication(application, gunicorn_options).run()

Oxysalt

Answer 2

I have just run into this problem myself, and the solution was to basically chain the commands to guarantee execution and correct order:

Script started by systemd, supervisord or any other such system:

#!/bin/sh
python manage.py my_custom_command && gunicorn project.wsgi $@

Create your own custom django command and off you go. You can get some speedups if you disable sanity checks in the command (requires_system_checks and requires_migrations_checks set to False).

To make things more generic, you could create a "boot" signal to which you connect arbitrary functions, and just emit the signal from this custom command.

Nebulose

Thank you for mentioning this workaround. However, I can't accept it as an answer, because the python manage.py environment can be substantially different from the gunicorn project.wsgi environment, and I want to run my checks in the latter. – Americanize

Answer 3

Gunicorn allows you to specify a variety of hooks, including on_starting and pre_fork, either of which sound like what you might want.

How to use: suppose you have the module my_app.startup_stuff that contains the following code you want to run:

def some_prefork_tasks(server):
  print('I\'m running pre-fork')

Create a gunicorn.conf.py file in the directory where you run gunicorn with the following contents:

from my_app import startup_stuff

pre_fork = startup_stuff.some_prefork_tasks

Then when you run gunicorn, it should print "I'm running pre-fork".

(gunicorn.conf.py is loaded automatically if it exists, but you can call the config file whatever and load it with -c <config>. See the docs on config files for more info.)

Dix