Trigering post_save signal only after transaction has completed

Question

I have written some APIs, for which the respective functions executive inside a transaction block. I am calling the save() method (after some modifications) on instance/s of a/several Model/s, and also consecutively indexing some JSON related information of the instance/s in Elasticsearch. I want the database to rollback even if for some reason the save() for one of the instances or indexing to the Elasticsearch fails.

Now, the problem is arising that even inside the transaction block, the post_save() signals gets called, and that is an issue because some notifications are being triggered from those signals.

Is there a way to trigger post_save() signals only after the transactions have completed successful?

Photothermic

Answer 1

Not really. The signals have nothing to do with the db transaction success or failure, but with the save method itself - before the call you have the pre_save signal fired and after the call you have the post_save signal fired.

There are 2 approaches here:

• you are going to inspect the instance in the post_save method and decide that the model was saved successfully or not; simplest way to do that: in the save method, after the transaction executed successfully, annotate your instance with a flag, say instance.saved_successfully = True, which you will test in the post_save handler.
• you are going to ditch the post_save signal and create a custom signal for yourself, which you will trigger after the transaction ran successfully.

Makes sense?

P.S.

If you strictly need to bind to the transaction commit signal, have a look over this package: https://django-transaction-hooks.readthedocs.org/en/latest/; it looks like the functionality is integrated in Django 1.9a.

Corona

Thanks, I had already tried the first approach, I was setting instance.in_transaction = True, but that did create a lot of the confusion as many of the other developers then missed the handling in their signals. I think, the second approach will be much more easy to handle, and will be with least complications. – Photothermic

Signals are included in transaction #36332253 – Pseudaxis

Answer 2

I think the simplest way is to use transaction.on_commit() (Django 2.1, Django 5.0). Here's an example using the models.Model subclass Photo that will only talk to Elasticsearch once the current transaction is over:

from django.db import transaction
from django.db.models.signals import post_save

@receiver(post_save, sender=Photo)
def save_photo(**kwargs):
    transaction.on_commit(lambda: talk_to_elasticsearch(kwargs['instance']))

Note that if the transaction.on_commit() gets executed while not in an active transaction, it will run right away.

Yongyoni

Thanks for sharing this. However, I was wondering how you were able to test the signal using django's test api. Test cases are wrapped in transactions that are always rolled back, resulting in the signal function never executing and the tests failing. – Chiccory

@Scratch'N'Purr If you use Django's unittest based framework, you can use TransactionTestCase, and if you use pytest, you can use the transactional_db fixture found in the pytest-django package! – Yongyoni

why do you use lambda? – Bankbook

@JuanDiegoRamirez We need to pass a function that takes no arguments into transaction.on_commit, so this was just the most concise way to express that. We could also just define a separate inline function like def _func(): talk_to_elasticsearch(kwargs['instance']) and use it like transaction.on_commit(_func) – Yongyoni

Answer 3

I was having serious issues with django's admin not allowing post_save transactions on parent objects when they had inline children being modified.

This was my solution to an error complaining about conducting queries in the middle of an atomic block:

def on_user_post_save_impl(user):
     do_something_to_the_user(user)

def on_user_post_save(sender, instance, **kwargs):
    if not transaction.get_connection().in_atomic_block:
        on_user_post_save_impl(instance)
    else:
        transaction.on_commit(lambda: on_user_post_save_impl(instance))

Tillotson

This does not work - AttributeError: "'module' object has no attribute 'on_commit'" – Rosariarosario

Have you imported transaction from django? I think it is django.db.transaction – Tillotson

why do you use lambda? – Bankbook

@JuanDiegoRamirez probably because I had a lambda at some point instead of a name function, and just never removed the lambda. It works. – Tillotson

Answer 4

EDIT: This answer is no longer relevant after on_commit was introduced in Django.

We are using this little nugget:

def atomic_post_save(sender, instance, **kwargs):
    if hasattr(instance, "atomic_post_save") and transaction.get_connection().in_atomic_block:
        transaction.on_commit(lambda: instance.atomic_post_save(sender, instance=instance, **kwargs))

post_save.connect(atomic_post_save)

Then we simply define a atomic_post_save method on any model we like:

class MyModel(Model):
    def atomic_post_save(self, sender, created, **kwargs):
        talk_to_elasticsearch(self)

Two things to notice:

1. We only call atomic_post_save when inside a transaction.
2. It's too late in the flow to send messages and have them included in the current request from inside atomic_post_save.

Kravits

why do you use lambda? – Bankbook

The lamda not is necesary docs.djangoproject.com/en/4.0/topics/db/transactions/… , i didn't vote negative on the contrary i will vote positive, take care bro – Bankbook

@JuanDiegoRamirez it truly is, read the docs you linked :) – Kravits

In the text says: "You can also wrap your function in a lambda", is optional, with function also it works, def do_something(): pass transaction.on_commit(do_something) – Bankbook

@JuanDiegoRamirez first argument to on_commit must be a callable, i.e. something that can be invoked inside on_commit. So either a reference to a function, a lambda or even a class instance that implements a __call__ method. See: https://mcmap.net/q/55129/-what-is-a-quot-callable-quot – Kravits

I ask you, is it strictly necessary to use lambda in on_commit? yes or no? – Bankbook

@JuanDiegoRamirez either lambda or partial is strictly needed if you need to pass parameters to callback function. – Nuriel