Thread Synchronization in Django

Question

Is there any way to block a critical area like with Java synchronized in Django?

Kaffiyeh

In many cases, doing something like Java's synchronized is not good enough for Django code. The usual scenario in a Django app is not to just protect a resource from other threads but from any code that will try to use the resource. In many cases, a WSGI server will use multiple processes to serve requests and thus you need to prevent not only other threads in a single process from accessing the resource but all processes that may try to access it. synchronized is not good enough for this and a Python solution that replicates what synchronized does won't be good enough either. – Azalea

Answer 1

You can use locks to make sure that only one Thread will access a certain block of code at a time.

To do this, you simply create a Lock object then acquire the lock before the block of code you want to synchronize. All the threads must have access to the same Lock object for this to work. An example:

from threading import Lock, Thread

lock = Lock()

def do_something():
    lock.acquire()   # will block if another thread has lock
    try:
        # ... use lock
    finally:
        lock.release()

Thread(target=do_something).start()
Thread(target=do_something).start()

For more information , see http://effbot.org/zone/thread-synchronization.htm.

Loosejointed

As Louis commented on the question, this will only work if there is a single server process, which is often not the case. – Styrene

Answer 2

My approach is to use the locking features of the database. This also works with multiple server processes.

I define a model as:

from django.db import models

class ThreadSafe(models.Model):
    key = m.CharField(max_length=80, unique=True)

And then a context manager function as:

from contextlib import contextmanager
from django.db.transaction import atomic

@contextmanager
def lock(key):
    pk = ThreadSafe.objects.get_or_create(key=key)[0].pk
    try:
        objs = ThreadSafe.objects.filter(pk=pk).select_for_update()
        with atomic():
            list(objs)
            yield None
    finally:
        pass

And then I have a thread/process safe lock by simply doing:

with lock("my_key"):
    do_scary_stuff_here()

This requires a database with support for transactions.

Auriol

The advantage with this approach is that this lock is shared by all processes (and all database connections) that connect to this database. But you pay a performance cost. – Impressure

Answer 3

Great article Justin, just one thing using python 2.5 makes this way easier

In Python 2.5 and later, you can also use the with statement. When used with a lock, this statement automatically acquires the lock before entering the block, and releases it when leaving the block:

from future import with_statement # 2.5 only

with lock: ... access shared resource

Serpasil

Answer 4

If you are using PostgreSQL, you could use advisory locks. Any process or thread could acquire or release this lock, assuming they are all connecting to the same PostgreSQL database. django-pglocks takes this approach.

Impressure