How to clear the whole cache when using django's page_cache decorator?

Question

I've got a pretty simple site where I'm using the page_cache decorator. I have a cronjob that checks for new data and processes it if it's available. (This is run using management commands executed with crontab)

I want to then clear all the page caches when new data is processed.

I'm looking at the documentation here: https://docs.djangoproject.com/en/stable/topics/cache/

and found cache.clear(), which seems to be what I want. I've added a flag to the data processing portion and execute cache.clear() when new data is found.

However, after the command runs the cache isn't cleared. (I've cleared the browser cached and checked to be sure it's not the browser)

Does cache.clear() not work in to clear all cached pages?

I'm using the DatabaseCache, so I guess I could go in and clear the cache table manually, but is there a better way?

Suzannsuzanna

Answer 1

I've had this problem with an SQLite database cache - the clear() method doesn't clear the cache although it works fine with a MySQL database cache. It seems that a SQLite cache needs a call to django.db.transation.commit_unless_managed() after the DELETE from [table] statement is run.

I have been using multiple caches since before official support was added into core as part of 1.3 and so have a wrapper round several of the cache calls - including clear() - so I was able to override this method and include the commit_unless_managed(). I think I should probably log it as a bug.

Here's the outline of the code I'm using to flush a memcache cache (the default cache in django.core.cache) and a database cache stored in the cache_table of the settings.DATABASES['cache_database'] database.

from django.db import connections, transaction
from django.core.cache import cache # This is the memcache cache.

def flush():
    # This works as advertised on the memcached cache:
    cache.clear()
    # This manually purges the SQLite cache:
    cursor = connections['cache_database'].cursor()
    cursor.execute('DELETE FROM cache_table')
    transaction.commit_unless_managed(using='cache_database')

Rather than being lazy and hard coding it the way I have it should be pretty easy to get the values from settings.CACHES and django.db.router.

Masseur

I ran into a rookie mistake here: cache.clear() did work but the browser displayed the locally cached version. Be sure to also enforce a proper page refresh when you debug this functionality. – Alessi

In FF needed to explicitly set Disable Cache in Network Tab – Curtin

Answer 2

It's a bug #19896 that looks to be fixed in 1.6.

If you are using an older version doing something like the following should make the clear work as expected.

from django.db import router, transaction


def clear_cache(the_cache):
    the_cache.clear()
    # commit the transaction
    db = router.db_for_write(the_cache.cache_model_class)
    transaction.commit_unless_managed(using=db)

This just makes sure that the transaction gets committed.

Afghan

Answer 3

Put something into cache, then try to call cache.clear() from the manage.py shell console and then manually check database cache contents. If that works then maybe your cache.clear() is just not called when new data is found.

The easiest way to understand what is going under the hood is just to put import pdb; pdb.set_trace() to the beginning of the cache.clear() function, then run debug server and wait, then some code call this func you'll be able to execute step-by-step its code or you'll just see that this func is not called as your expected.

Sontag

I did check with the shell, and even when I run cache.clear() from the shell it doesn't seem to clear the cache. It does seem to get cleared based on the configured expire period however... – Suzannsuzanna

The clear method of database cache backend is quite simple: code.djangoproject.com/browser/django/tags/releases/1.3/django/… It just delete all records from the cache table. Are you sure your cache is configured correctly? Maybe you use memcached or something else? Other reason could be browser cache, to be sure you can open your website in other browser which you have not used yet to look that site. – Sontag

Answer 4

You can clear cache (delete all cache values) with a custom command. *I use LocMemCache which is used by default in Django:

For example, create clearcache.py in core/management/commands/ as shown below. *Don't forget to put __init__.py to management and commands folders and the doc explains how to create a custom command in Django:

django-project
 |-core
 |  |-settings.py
 |  └-management
 |     |-__init__.py
 |     └-commands
 |        |-__init__.py
 |        └-clearcache.py # Here
 |-my_app1
 └-my_app2

Because you put clearcache.py in core folder where settings.py is so you need to set 'core' to INSTALLED_APPS otherwise the custom command is not recognized by Django:

# "core/settings.py"

INSTALLED_APPS = [
    ...
    'core', # Here
    ...
]

Then, create Command class extending BaseCommand class in core/management/commands/clearcache.py as shown below:

# "core/management/commands/clearcache.py"

from django.core.management.base import BaseCommand
from django.core.cache import cache

class Command(BaseCommand):
    def handle(self, *args, **kwargs):
        cache.clear()
        self.stdout.write('Cache is cleared\n')

Finally, running the command below can clear cache:

$ python manage.py clearcache
Cache is cleared

Pepsinate