Running Django migrations in a multi-container Docker setup

Question

Is it safe to allow multiple instances of a Django application to run the same database migration at the same time?

Scenario description

This is a setup where a multiple instances of a Django application are running behind a load balancer. When an updated version of the Docker container is available, each of the old Docker images are replaced with the new version.

If new Django migrations exist, they need to be run. This leads me to the question: is it safe to allow multiple containers to run the migration (python manage.py migrate) at the same time?

---

I have two hypothesis about what the answer to this question might be.

1. Yes it is safe. Due to database level locking, the migration's can't conflict and in the end, one migration script will run while the other reports that there are no migrations to apply.
2. No this is not safe. The two migrations can possibly conflict with each other as they try to modify the database.

Unreality

Answer 1

No, it's not safe to run the migration in all containers at the same time, since you might end up applying the same migration twice.

There are two possible cases:

1. Applying the migration twice (e.g. adding a table column) violates a database constraint, so therefore only the first container that run the migration manages to finish the migration. In this case the other containers will die, although your orchestration system probably will restart them.

2. Applying the migration twice doesn't violate any constraint and therefore can be applied multiple times. In this case you can end up with duplicated data.

In any case, you should try to have only one container applying migrations at the same time.

Hurlyburly