Execute code when Django starts ONCE only?

Question

I'm writing a Django Middleware class that I want to execute only once at startup, to initialise some other arbritary code. I've followed the very nice solution posted by sdolan here, but the "Hello" message is output to the terminal twice. E.g.

from django.core.exceptions import MiddlewareNotUsed
from django.conf import settings

class StartupMiddleware(object):
    def __init__(self):
        print "Hello world"
        raise MiddlewareNotUsed('Startup complete')

and in my Django settings file, I've got the class included in the MIDDLEWARE_CLASSES list.

But when I run Django using runserver and request a page, I get in the terminal

Django version 1.3, using settings 'config.server'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
Hello world
[22/Jul/2011 15:54:36] "GET / HTTP/1.1" 200 698
Hello world
[22/Jul/2011 15:54:36] "GET /static/css/base.css HTTP/1.1" 200 0

Any ideas why "Hello world" is printed twice? Thanks.

Chatelaine

just for curiosity, did you figured why the code in init.py gets executed twice? – Annoyance

@Annoyance it only gets executed twice under runserver ... that is because runserver first loads up the apps to inspect them and then actually starts the server. Even upon autoreload of runserver the code is only exec once. – Whitford

Wow I have been here.... so thank you again for the comment @Pykler, that is what I was wondering. – Matins

@Whitford @Annoyance it only gets executed twice under runserver you mean once.. – Balinese

Answer 1

Update from Pykler's answer below: Django 1.7 now has a hook for this

---

Don't do it this way.

You don't want "middleware" for a one-time startup thing.

You want to execute code in the top-level urls.py. That module is imported and executed once.

urls.py

from django.confs.urls.defaults import *
from my_app import one_time_startup

urlpatterns = ...

one_time_startup()

Expeller

Just to be sure, will this work with Django on top of apache/mod_wsgi? Or just with the django runsrever command? – Justino

@Michael: They're effectively identical (except for speed and overall "security"). Yes. It works under Apache and mod_wsgi. – Expeller

What would you use if you need to run startup code before management commands as well? – Queeniequeenly

@Andrei: Management Commands are entirely a separate problem. The idea of special one-time startup before all management commands is hard to understand. You'll have to provide something specific. Perhaps in another question. – Expeller

@S.Lott: Yes, you are right. On the other hand, settings are run for each command and it would be nice to have a general solution for the problem which would work in all cases. In my case, I have to run mongoengine connection in settings.py, what I try to avoid. – Queeniequeenly

@S.Lott: BTW, here is my question #9260344 – Queeniequeenly

Tried printing simple text in urls.py, but there was absolutely no output. What is happening ? – Manutius

Be sure if you are using mod_wsgi to set it to deamon mode and if needed run it as a single process. See more here: code.google.com/p/modwsgi/wiki/IntegrationWithDjango keyword: WSGIDaemonProcess. – Honorine

The urls.py code is executed only at first request (guess it answers @SteveK 's question) (django 1.5) – Meraz

This executes once for each worker, in my case, it's executed 3 times in total. – Poppyhead

and where do we define the function? in my_app/models.py? – Abstemious

@halilpazarlama This answer is out of date -- you should be using the answer from Pykler. – Cordoba

This can be a problem if you only want the code to run when running a server and not, say, when running tests, if you have tests that import urls – Uta

Also, when you run both ./manage runserver and ./manage.py rundramatiq, you get two executions if you use the urls.py approach. – Checkbook

@MarkChackerian the answer from Pykler doesn't work if there is an access to database – Mufti

Answer 2

Update: Django 1.7 now has a hook for this

file: myapp/apps.py

from django.apps import AppConfig
class MyAppConfig(AppConfig):
    name = 'myapp'
    verbose_name = "My Application"
    def ready(self):
        pass # startup code here

file: myapp/__init__.py

default_app_config = 'myapp.apps.MyAppConfig'

For Django < 1.7

The number one answer does not seem to work anymore, urls.py is loaded upon first request.

What has worked lately is to put the startup code in any one of your INSTALLED_APPS init.py e.g. myapp/__init__.py

def startup():
    pass # load a big thing

startup()

When using ./manage.py runserver ... this gets executed twice, but that is because runserver has some tricks to validate the models first etc ... normal deployments or even when runserver auto reloads, this is only executed once.

Whitford

In the latest version of django this executes only once with manage.py commands. I am not even entirely sure if its because of the new version of django or what, but basically even when it executed twice, the first time was just a check, the second time is when it counts. – Whitford

I think this gets executed for each process that loads the project. So, I can't think of why this wouldn't work perfectly under any deployment scenario. This does work for management commands. +1 – Peck

Note though ... Under wsgi servers (atleast uwsgi) it does lazy loading of the app which means this will get triggered on first request. Putting a call to import myapp int he wsgi.py is what I ended up doing, but it does not always work! it fails if you are starting things like subprocess with pipes (yeah deadly right) that cannot be shared with other processes (since uwsgi forks the original process in a multi worker model. – Whitford

PS tests show that with Django 1.7 this only gets executed on first request. Updating wsgi.py is also needed to trigger startup before the first request. – Whitford

I understand that this solution can be used to execute some arbitrary code when the server starts but is it possible to share some data that would be loaded? For example, I want to load an object that contains a huge matrix, put this matrix in a variable and use it, via a web api, in each request a user can do. Is such a thing possibe? – Fastidious

@Fastidious yes you can, in your startup function can look like this gist.github.com/pykler/024334b23f18d66937f2 – Whitford

can't not create a initial model in this way – Jordan

The documentation says this is not the place to do any database interaction. That makes it unsuitable for a lot of code. Where could this code go? – Slaughterhouse

@Slaughterhouse you have to read that warning with a little more depth. It doesn't say do not do it, just explains that you have to know what you are doing. For example, reading from the db is totally fine but writing to it, might not be wise in many cases. That being said, who writes to a database every time their app restarts, doesn't seem wise at all ... right? – Whitford

Doesn't work for me. I'm starting the server with python3 manage.py runserver, also I'm not using a production server. – Corded

@Whitford Is there anything similar for the AppConfig.ready() method? I mean initiating an object that will later be importable and usable by various modules in the app. Thanks. – Tollefson

@Tollefson not sure I follow, but in the ready method you can create anything you want and can make them module level globals that are importable if thats what you need. – Whitford

@Whitford I solved my problem by initiating a member of AppConfig-inheriting class. Thanks. – Tollefson

@Whitford unfortunatly the ready() function is also called each times makemigrations is called. Is there a way to make this function called only for runserver. – Donall

EDIT: A possible hack is to check the command lines arguments any(x in sys.argv for x in ['makemigrations', 'migrate']) – Donall

I'm getting django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet. on Django 1.10 when trying this, is this answer out of date now? – Convent

I'm also getting AppRegistryNotReady when trying to load a model from the data store. – Sunflower

If your script is running twice you check out this answer: https://mcmap.net/q/50199/-why-is-run-called-twice-in-the-django-dev-server – Hectare

This gets executed after parsing the settings. Any idea how to make this load before the settings? – Populace

@chefarov not sure what you are trying to do, seems complex. You would have to dig into django's startup sequence to see where best to put your code. Most likely though, easiest is to call it at the top of settings file itself if you want to make sure the settings have not been loaded yet. – Whitford

@Whitford Thanks for your comment. That's what I did actually. I wanted to create empty logs folder before logging.config.dictConfig() was called. Finally I put it before that call. – Populace

Pah, that's really not easy. In settings.py you can put code that has to run before everything else, e.g dynamically adding apps. But you have no models there, and even no logger neither. apps.AppConfig.ready() is not possible for writing to DB, and slows down development, as it is run at runserver twice, and at every other management command as well. So well, no-go too. I think one of the best options is to write a middleware, or (ab)use urls.py – Solmization

My project is called foo, with settings.py in foo/foo/settings.py and various apps in foo/apps/app_a/..., foo/apps/app_b/..., etc. runfirst() acts on all apps in INSTALLED_APPS. Where exactly do I place my runfirst() code? – Booklover

@Booklover you would be better off splitting your run_first into one for each app. You might need to do some refactoring to make the code be able to run in any order, since there is no real order in which each app's setup runs. To be honest, not sure if there will be a standard solution to what you are asking since apps are supposed to be independant of each other. One quick answer would be pick any of the apps and put your runfirst() function call there. That would solve your problem. – Whitford

It should be noted that the database can be accessed using this as long as the imports are loaded after the ready method is called: #54056315 . – Heterogynous

default_app_config = 'myapp.apps.MyAppConfig' is not needed as of Django 3.2. From the docs: Changed in Django 3.2: In previous versions, a default_app_config variable in the application module was used to identify the default application configuration class. – Disarrange

This is explicitly discouraged by the docs. The reason is if you run tests, the ready() code will still run against your production database, not the test database. See discussion here. – Ligulate

We have lots of applications in our Django app and each one of these has a AppConfig, will inserting it under one of them work for all the others? Context: I want to initialise GCP Profiler – Sufficiency

Answer 3

This question is well-answered in the blog post Entry point hook for Django projects, which will work for Django >= 1.4.

Basically, you can use <project>/wsgi.py to do that, and it will be run only once, when the server starts, but not when you run commands or import a particular module.

import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "{{ project_name }}.settings")

# Run startup code!
....

from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

Costello

Again adding a comment to confirm that this method will execute the code only once. No need for any locking mechanisms. – Disburse

Scripts have been added here seem not be executed when the test framework starts – Aplacental

This answer ended a two and a half day search for solutions that simply didn't work. – Beechnut

Note that this executes when the first request is made to the website, not when you start Apache. – Antepenult

Just in case. This solution will not work when running the dev server via manage.py runserver. – Sirkin

Answer 4

As suggested by @Pykler, in Django 1.7+ you should use the hook explained in his answer, but if you want that your function is called only when run server is called (and not when making migrations, migrate, shell, etc. are called), and you want to avoid AppRegistryNotReady exceptions you have to do as follows:

file: myapp/apps.py

import sys
from django.apps import AppConfig

class MyAppConfig(AppConfig):
    name = 'my_app'

    def ready(self):
        if 'runserver' not in sys.argv:
            return True
        # you must import your modules here 
        # to avoid AppRegistryNotReady exception 
        from .models import MyModel 
        # startup code here

Reamy

does this run in production mode? AFAIK in prod. mode there is no "runserver" started. – Solmization

Thanks for this! I have Advanced Python Scheduler in my app and I didn't want to run the scheduler when running manage.py commands. – Wira

Do you need to run ready() at some point ? – Mufti

Answer 5

If it helps someone, in addition to pykler's answer, "--noreload" option prevents runserver from executing command on startup twice:

python manage.py runserver --noreload

But that command won't reload runserver after other code's changes as well.

Akkerman

Thanks this solved my problem! I hope when I deploy this doesn't happen – Humane

As an alternative, you can check the content of os.environ.get('RUN_MAIN') to only execute your code once in the main process (see stackoverflow.com/a/28504072) – Mount

Yup, this plus pykler's answer worked for me also, as it prevented the multiple ready(self) calls while still being able to start them only once. Cheers! – Fakieh

Django's runserver by default starts two processes with distinct (different) pid numbers. --noreload makes it start one process. – Checkbook

Answer 6

Standard solution

With Django 3.1+ you can write this code to execute only once a method at startup. The difference from the other questions is that the main starting process is checked (runserver by default starts 2 processes, one as an observer for quick code reload):

import os 
from django.apps import AppConfig

class MyAppConfig(AppConfig):
    name = 'app_name'

    def ready(self):
        if os.environ.get('RUN_MAIN'):
            print("STARTUP AND EXECUTE HERE ONCE.")
            # call here your code

Another solution is avoiding the environ check but call --noreload to force only one process.

Alternative method

The first question to answer is why we need to execute code once: usually we need to initialize some services, data in the database or something one-shot. The 90% of the time it is some database initialization or job queue.

The approach to use the AppConfig.ready() method is not reliable, not always reproducible in production and it cannot guarantee to be executed exactly once (but at least once that is not the same). To have something quite predictable and executed exactly one time the best approach is developing a Django BaseCommand and call it from a startup script.

So for example, we can code in your "myapp", in the file "app/management/commands/init_tasks.py":

from django.core.management.base import BaseCommand
from project.apps.myapp.tasks import scheduler
from project import logger, initialize_database_data

class Command(BaseCommand):
    help = "Init scheduler or do some staff in the database."

    def handle(self, *args, **options):
        scheduler.reload_jobs()
        initialize_database_data()
        logger.info("Inited")

And finally we can have a start script "Start.bat" (in the example a windows batch) to setup the full application start:

start /b python manage.py qcluster
start /b python manage.py runserver 0.0.0.0:8000
start /b python manage.py init_tasks

Tisza

The BaseCommand answer works best for me! And I think it should be the accepted answer. – Septime

excellent!!! I think it should be the accepted answer, too. – Complain

Works fine as well on Linux. Just make sure to put management/commands/init_tasks.py inside an app, not the base package. – Baese

Answer 7

Note that you cannot reliability connect to the database or interact with models inside the AppConfig.ready function (see the warning in the docs).

If you need to interact with the database in your start-up code, one possibility is to use the connection_created signal to execute initialization code upon connection to the database.

from django.dispatch import receiver
from django.db.backends.signals import connection_created

@receiver(connection_created)
def my_receiver(connection, **kwargs):
    with connection.cursor() as cursor:
        # do something to the database

Obviously, this solution is for running code once per database connection, not once per project start. So you'll want a sensible value for the CONN_MAX_AGE setting so you aren't re-running the initialization code on every request. Also note that the development server ignores CONN_MAX_AGE, so you WILL run the code once per request in development.

99% of the time this is a bad idea - database initialization code should go in migrations - but there are some use cases where you can't avoid late initialization and the caveats above are acceptable.

Renin

This is a good solution if you need to access the database in your startup code. A simple method to get it to run only once is to have the my_receiver function disconnect itself from the connection_created signal, specifically, add the following to the my_receiver function: connection_created.disconnect(my_receiver). – Dysentery

Answer 8

if you want print "hello world" once time when you run server, put print ("hello world") out of class StartupMiddleware

from django.core.exceptions import MiddlewareNotUsed
from django.conf import settings

class StartupMiddleware(object):
    def __init__(self):
        #print "Hello world"
        raise MiddlewareNotUsed('Startup complete')

print "Hello world"

Guitarfish

Hi Oscar! On SO, we prefer that answers include an explanation in English, and not just code. Could you please give a brief explanation of how/why your code fixes the problem? – Brachycephalic

Answer 9

In my case, I use Django to host a site, and using Heroku. I use 1 dyno (just like 1 container) at Heroku, and this dyno creates two workers. I want to run a discord bot on it. I tried all methods on this page and all of them are invalid.

Because it is a deployment, so it should not use manage.py. Instead, it uses gunicorn, which I don't know how to add --noreload parameter. Each worker runs wsgi.py once, so every code will be run twice. And the local env of two workers are the same.

But I notice one thing, every time Heroku deploys, it uses the same pid worker. So I just

if not sys.argv[1] in ["makemigrations", "migrate"]: # Prevent execute in some manage command
    if os.getpid() == 1: # You should check which pid Heroku will use and choose one.
        code_I_want_excute_once_only()

I'm not sure if the pid will change in the future, hope it will be the same forever. If you have a better method to check which worker is it, please tell me.

Wilie

Answer 10

I used the accepted solution from here which checks if it was run as a server, and not when executing other managy.py commands such as migrate

apps.py:

from .tasks import tasks

class myAppConfig(AppConfig):
    ...

    def ready(self, *args, **kwargs):
        is_manage_py = any(arg.casefold().endswith("manage.py") for arg in sys.argv)
        is_runserver = any(arg.casefold() == "runserver" for arg in sys.argv)

        if (is_manage_py and is_runserver) or (not is_manage_py):
            tasks.is_running_as_server = True

And since that will still get executed twice when in development mode, without using the parameter --noreload, I added a flag to be triggered when it is running as a server and put my start up code in urls.py which is only called once.

tasks.py:

class tasks():
    is_running_as_server = False

    def runtask(msg):
        print(msg)

urls.py:

from . import tasks

task1 = tasks.tasks()

if task1.is_running_as_server:
    task1.runtask('This should print once and only when running as a server')

So to summarize, I am utilizing the read() function in AppConfig to read the arguments and know how the code was executed. But since in the development mode the ready() function is run twice, one for the server and one for the reloading of the server when the code changes, while urls.py is only executed once for the server. So in my solution i combined the two to run my task once and only when the code is executed as a server.

Fiden

Answer 11

For those looking to only run their code once on startup in a production environment using gunicorn, you can make use of the the --preload command given by gunicorn.

By default Django uses multiple workers for any startup code, but by passing the --preload command Django will only run the startup command in the parent worker.

This is a good post explaining how to add the --preload command. Then just add your code to run on startup under the ready function like belo

from django.apps import AppConfig

class MyAppConfig(AppConfig):
    name = 'app_name'

    def ready(self):
       code_to_run()

Goingover