best place to clear cache when restarting django server

Question

I want memcached to be flushed on every restart/reload of django server. I use cherrypy for production and builtin server for development.

I would add this to settings.py, right after CACHES:

from django.core.cache import cache
cache.clear()

but it makes a recursive import:

Error: Can't find the file 'settings.py' in the directory containing 'manage.py'. It appears you've customized things.
You'll have to run django-admin.py, passing it your settings module.
(If the file settings.py does indeed exist, it's causing an ImportError somehow.)
make: *** [server] Error 1

Any other suggestions? Thanks.

Berck

Answer 1

It's bad practice to put code in settings.py other than assignments. It's better suited as a management command:

from django.core.management.base import BaseCommand
from django.core.cache import cache

class Command(BaseCommand):
    def handle(self, *args, **kwargs):
        cache.clear()
        self.stdout.write('Cleared cache\n')

Which you can add to your project by sticking it in someapp/management/commands. For instance you could create a new app called utils and add that to your INSTALLED_APPS and the directory structure would look like this:

utils
├── __init__.py
└── management
    ├── __init__.py
    └── commands
        ├── __init__.py
        └── clearcache.py

You can now clear cache by doing ./manage.py clearcache. If you want to run clearcache every time you runserver you can just write a shell alias to do that:

alias runserver='./manage.py clearcache && ./manage.py runserver'

Alternately I think you can write it as a stand-alone script and configure the settings it requires by hand:

from django.conf import settings

# obviously change CACHES to your settings
CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
        'LOCATION': 'unique-snowflake'
    }
}

settings.configure(CACHES=CACHES) # include any other settings you might need

from django.core.cache import cache
cache.clear()

Writing your stand-alone script like this will prevent circular imports, and allow you to import it from your settings.py. Although there is no guarantee that settings.py will be imported only once, so in general I'd avoid this. It'd be nice if the signal framework could fire off an event once every time the app is started, after settings are loaded for stuff like this.

Touching

Does not work when I HUP the server externally, I want it to call the clear cache no matter how I restart/reload the server. I use runit in production to start/stop cherrypy. In that case, your suggestion would not work. I want the solution to be usable for different setups. – Spirited

I don't know what runit is, but you can use the management command at any time. – Touching

I'm assuming that you have some sort of init script? Just add a call to clearcache to that. – Touching

It just kills and restarts the django server. I agree with you, but I want to make clearing the cache independent on system setup. As I am not the only developer and >1 setup exists (production, development and testing). All have different management and django service invocation. – Spirited

@Motiejus - if you have a script that just "kills and restarts the server", add a line underneath that calls the management command. This is the "correct" way to do what you're trying to do. Other approaches put code where it doesn't belong (e.g. @freakish's approach), which is significantly worse long term than asking your developers to run a script to restart the server. – Quirk

@Dominic: surely my solution is worse, then zeekay's and I wouldn't recommend it. But still it is the answer for the question, right? :) Unlike zeekay's. – Unblushing

@Unblushing @Motiejus To clarify, the second part of my answer shows how you can import a stand-alone script to clear your cache in settings.py and avoid circular imports – Touching

Worth bearing in mind that if you have your sessions tied to your cache, this will log out all your users. – Cosmogony

Answer 2

Django Extensions lets you wipe cache via

manage.py clear_cache

more info and many further commands in their docs.

Kantianism

Answer 3

You typically only wan't to invalidate your caches if the code changes in a way that requires a new cache. Not on every restart.

This is best handled by using the Django feature: settings.CACHES['VERSION'][1], and increase that number every time you change the code that changes the format of cached data. That way, on a deploy, you automatically will use a fresh cache when you deploy new code, but keep the cache if you're code is cache-compatible with the previous code.

[1]: https://docs.djangoproject.com/en/stable/ref/settings/#std:setting-CACHES-VERSION)

Isoniazid

Answer 4

How about this? Define a boolean value in settings.py, for example CLEAR_CACHE_ON_RESTART = True and then check in other place if it is True. If it is, then clear cache and set it to False. This code can be placed in any view (like a main view) and probably even in manage.py or urls.py (although I didn't check this and it doesn't look too good). Give it a try!

Unblushing

It is tempting to put the code mentioned to somewhere, which is loaded at loading time (urls.py, views.py, models.py of any application). However, I want to be sure it's loaded only once per run-time. Not once per thread or per request. For that reason your suggestion is unacceptable. – Spirited

Why not? As I said, you need to check for boolean global constant (define it in settings) and set it to false after first check. All other checks will do nothing. Also note, that (I think) urls.py is loaded only once, after server restart (although I didn't check this). Still, this is NOT the right way to do things. To do this right, you really should write some external script. – Unblushing

So they will have to switch between False to True everytime he want to reload/restart the server? – Cydnus

Answer 5

If you have multiple cache backends, django.core.cache.cache.clear() will only clear the default cache. To make sure your clear_cache command clears cache from all your backends, you should use the following command:

from django.conf import settings
from django.core.cache import caches

from django.core.management.base import BaseCommand

class Command(BaseCommand):
    help = "Clear cache"

    def handle(self, **options):
        for k in settings.CACHES.keys():
            caches[k].clear()
            self.stdout.write("Cleared cache '{}'.\n".format(k))

Sieve