How to create per-project initial_data fixtures in Django 1.7+

Question

Before Django 1.7 I used to define a per-project fixtures directory in the settings:

FIXTURE_DIRS = ('myproject/fixtures',)

and use that to place my initial_data.json fixture storing the default groups essential for the whole project. This has been working well for me as I could keep the design clean by separating per-project data from app-specific data.

Now with Django 1.7, initial_data fixtures have been deprecated, suggesting to include data migrations together with app's schema migrations; leaving no obvious choice for global per-project initial data.

Moreover the new migrations framework installs all legacy initial data fixtures before executing migrations for the compliant apps (including the django.contrib.auth app). This behavior causes my fixture containing default groups to fail installation, since the auth_group table is not present in the DB yet.

Any suggestions on how to (elegantly) make fixtures run after all the migrations, or at least after the auth app migrations? Or any other ideas to solve this problem? I find fixtures a great way for providing initial data and would like to have a simple and clean way of declaring them for automatic installation. The new RunPython is just too cumbersome and I consider it an overkill for most purposes; and it seems to be only available for per-app migrations.

Linette

Since it's deprecated it's probably gonna be a bit awkward. One option might be to create a minimal app for loading the initial data only, create a migration, and in RunPython, call whatever functions Django uses for loading fixtures. That way you could also specify dependencies in your migration. It would probably involve some serious digging in the Django source code though. – Pietro

Yeah, I've though about creating some kind of meta application as a last resort and creating the migrations there (but it's still ugly). Wonder why the new Django does not at least provide a standard way of declaratively specifying fixtures to be loaded together with the new Migrations. – Linette

Somewhere on SO it was suggested to reuse the pre-1.7 json/xml fixture files. You would have a minimal app with just a data migration, then parse your existent json/xml fixtures and create records from them. This should be what happened pre-1.7 anyways. – Jordanjordana

Answer 1

If you absolutely want to use fixtures, just use RunPythonand call_command in your data migrations.

from django.db import migrations
from django.core.management import call_command

def add_data(apps, schema_editor):
    call_command('loaddata', 'thefixture.json')

def remove_data(apps, schema_editor):
    call_command('flush')


class Migration(migrations.Migration):

    dependencies = [
        ('roundtable', '0001_initial'),
    ]

    operations = [
        migrations.RunPython(
            add_data,
            reverse_code=remove_data),
    ]

However this is recommanded to load data using python code and Django ORM, as you won't have to face integrity issues.

Source.

Takao

This is the correct answer, and appears to come from this excellent discussion on the topic. andrewsforge.com/article/upgrading-django-to-17/… – Lucrece

I just watched the talk by Andrew Pinkham who explains it nicely, so this really seems to be the way to go. As he says: "The initial_data system is a problem when used in tandem with migrations"; so that's why "The initial_data system is gone, but the fixtures aren't". – Linette

warning: 'flush' command as shown here will remove all data, not just the data from the forwards migration – Heisenberg

Answer 2

I recommend using factories instead of fixtures, they are a mess and difficult to maintain, better to use FactoryBoy with Django.

Stasiastasis

I don't think fixtures are difficult to maintain, they've been a quite favourite tool so far, and they are a built-in feature. I'm not that comfortable using a 3rd party app like FactoryBoy for such a simple task as loading data. I think it's an overkill for this purpose. After all, FactoryBoy is intended for complex unit testing, as an advanced data generator. That's far behind the scope of my purpose - simple static data loader. But thanks for the tip anyway. – Linette