Django migrate --fake and --fake-initial explained

Question

I've been a user of Django for about 2 years now and there is a feature I have always been afraid of using : faking migrations.

I've looked pretty much everywhere and the most information I can get is from the documentation where it states that:

--fake

Tells Django to mark the migrations as having been applied or unapplied, but without actually running the SQL to change your database schema.

This is intended for advanced users to manipulate the current migration state directly if they’re manually applying changes; be warned that using --fake runs the risk of putting the migration state table into a state where manual recovery will be needed to make migrations run correctly.

--fake-initial

Allows Django to skip an app’s initial migration if all database tables with the names of all models created by all CreateModel operations in that migration already exist. This option is intended for use when first running migrations against a database that preexisted the use of migrations. This option does not, however, check for matching database schema beyond matching table names and so is only safe to use if you are confident that your existing schema matches what is recorded in your initial migration.

I get the general idea and why one would want to use this feature. But, I don't understand the part where it says that this is intended for advanced users only.

Can someone explain what is happening behind the scene and why manual recovery would be needed.

NOTE

I'm not looking for the exact raw SQL queries that runs when faking a migration. I'm only looking for a general idea of what is happening behind the scene and maybe an example of why faking a migration would result in a state where makemigrations would not be working correctly.

Plum

I think it's worth to mention that when you run --fake, marking migrations as applied or not, is defined at django_migrations table, where Django keeps track of all applied migrations for an app, with the name of the migration file and when it was applied. It took me a while to figure this, since that the documentation isn't clear about this detail which I presented here. – Shebashebang

Answer 1

It is related to a database problem similar to a merge conflict in a source code (git) if you need to combine two branches with similar models or to switch between them. Nobody likes it intentionally.

Imagine that you started to modify an application last week, maybe because you found a bug or you extended the app by a field or table. Today you received an update and you have a problem, because there is a migration that adds a field that is still in your database and you can apply only other parts of that migration. You look at SQL contents of the migration by running

./manage sqlmigrate some_app 0007_new_migration >customized-some_app-0007_new_migration.sql

compare the content with the change made last week and remove or comment out a command that is still applied and can not be repeated. Run all remaining SQL manually. Mark that migration like it would be applied automatically:

./manage migrate --fake some_app 0007_new_migration

If you break something, nobody can help you probably, because the migration system will not know the current state of the database more. Therefore do a backup, write notes, use a sandbox and work precisely.

EDIT: The migration table django_migrations is a simple list of migrations applied in all apps. Rows in this table should be always in a synchronized status with the database structure. Migrations can be applied by a normal migrate. (or un-applied by a reverse migration to an older state, usually with some data loss of course) A fake migration applies the change only to the django_migrations table.

me => select * from django_migrations;
 id | app      |          name           |            applied            
----+----------+-------------------------+-------------------------------
  1 | some_app | 0001_initial            | 2017-10-16 06:11:07.31249+02
  2 | some_app | 0002_auto_20171016_1905 | 2017-10-17 02:05:48.979295+02

A migration (file) is a description of incremental change and also information to be possible to evaluate the difference between models.py since the last migration, that is compared when running makemigrations. It is enough also in the case where some tables were un-managed initially and they could become managed later. (therefore unmanaged tables are also recorded.)

EDIT: An example: how sqlmigrate with --fake could be used to fix a broken database by migrations (to recreate a deleted table).

EDIT: Example: If you decided to delete tables of some application and to create them again by migrate (be warned and see my comment below), you probably also want to reset all migrations of that application first, including the initial migration, by a pseudo-migration name "zero".
./manage migrate --fake some_app zero.

Godewyn

Thanks for the answer! It helped me seeing a real-life situation for faking migrations, but I'm still wondering what is happening behind the scene while faking migrations. – Plum

@Godewyn I've got question for your last EDIT, why would I want to "reset all migrations" when they are historicaly synchronized with my database? Recently I had similar issue with adding unique attirbute to one of my row in table. When I ran migrations I recived error: "ProgrammingError: relation "...unique" already exists", but when I deleted all of my migrations and replayed this step, it passed? I must point out that my two projects are working on the same database and I'm adding unique attribute in both apps. – Meingoldas

@Meingoldas I should add some warning to the last example "and you know what you are doing" e.g. if it is in development without important data and synchronization is so much broken that no forward or backward migration is not possible then a fake migration to "zero" and delete some tables could be a better solution then a slow editing of results of sqlmigrate and also better than a fast delete of whole database. – Godewyn

If your two projects work on the same database then it is important to use the same models.py and therefore the same migrations directory. What is migrated in one project that will be migrated also in the second project. If common models are not possible then if could be terrible and you can never run migrations in the second database and run manually only small parts of sqlmigrate and to set also Meta: managed=False. – Godewyn

In fact, there are a lot of situations you need faking migrations. especially when you want to use a db dump from one server in another server. faking is really useful when you encounter bugs that are hard to debug and cannot reproduce except in the production server. Messing up migration files beacause of git conflicts is because of a misconfigured .gitignore file. It is not a good example as a use case of faking migrations – Clinker