Django loaddata UNIQUE constraint failed

Question

I'm running python manage.py loaddata 'path/to/mydata.json' with an empty database (User and UserProfile tables are created but not populated), however, I'm getting the following error:

django.db.utils.IntegrityError: Problem installing fixture 'path/to/mydata.json': Could not load myapp.UserProfile(pk=1): UNIQUE constraint failed: myapp_userprofile.user_id

I checked (even after running this command) and the database is not populated at all. So how can it be giving an error that the pk is not unique?

If relevant, UserProfile just extends the default User model with a OneToOneField relation, as proposed here.

Here is what mydata.json contains:

[
    {
        "model": "auth.user",
        "pk": 1,
        "fields": {
            "password": "pbkdf2_sha256..",
            "last_login": "2016-10-22T15:19:46.926Z",
            "is_superuser": true,
            "username": "thesuperuser",
            "first_name": "",
            "last_name": "",
            "email": "[email protected]",
            "is_staff": true,
            "is_active": true,
            "date_joined": "2016-10-22T14:48:27.394Z",
            "groups": [],
            "user_permissions": []
        }
    },
    {
        "model": "auth.user",
        "pk": 2,
        "fields": {
            "password": "pbkdf2_sha256..",
            "last_login": null,
            "is_superuser": false,
            "username": "user1",
            "first_name": "User",
            "last_name": "One",
            "email": "",
            "is_staff": false,
            "is_active": true,
            "date_joined": "2016-10-22T15:20:32Z",
            "groups": [],
            "user_permissions": []
        }
    },
    {
        "model": "auth.user",
        "pk": 4,
        "fields": {
            "password": "pbkdf2_sha256..",
            "last_login": null,
            "is_superuser": false,
            "username": "user3",
            "first_name": "User",
            "last_name": "Three",
            "email": "",
            "is_staff": false,
            "is_active": true,
            "date_joined": "2016-10-22T15:21:09Z",
            "groups": [],
            "user_permissions": []
        }
    },
    {
        "model": "auth.user",
        "pk": 3,
        "fields": {
            "password": "pbkdf2_sha256..",
            "last_login": null,
            "is_superuser": false,
            "username": "user2",
            "first_name": "User",
            "last_name": "Two",
            "email": "",
            "is_staff": false,
            "is_active": true,
            "date_joined": "2016-10-22T15:21:03Z",
            "groups": [],
            "user_permissions": []
        }
    },
    {
        "model": "myapp.userprofile",
        "pk": 1,
        "fields": {
            "user": 1,
            "money": 100
        }
    },
    {
        "model": "myapp.userprofile",
        "pk": 2,
        "fields": {
            "user": 2,
            "money": 100
        }
    },
    {
        "model": "myapp.userprofile",
        "pk": 3,
        "fields": {
            "user": 3,
            "money": 100
        }
    },
    {
        "model": "myapp.userprofile",
        "pk": 4,
        "fields": {
            "user": 4,
            "money": 100
        }
    }
]

Thanks for any help,

Phenoxide

Answer 1

Exclude ContentType and Auth Permissions objects when creating a db dump.

python manage.py dumpdata --natural-foreign --natural-primary -e contenttypes -e auth.Permission --indent 2 > dump.json

After that you should be able to run the command without any issue

python manage.py loaddata dump.json

Credit to https://www.coderedcorp.com/blog/how-to-dump-your-django-database-and-load-it-into-/ for saving my day

Cheremkhovo

Answer 2

Today (24th, April, 2020) I had a similar problem with Django 2.2

My fixtures file was as simple as this:

[
{
    "model": "contenttypes.contenttype",
    "pk": 1,
    "fields": {
        "app_label": "admin",
        "model": "logentry"
    }
},
{
    "model": "contenttypes.contenttype",
    "pk": 2,
    "fields": {
        "app_label": "auth",
        "model": "permission"
    }
}]

When I ran ./manage.py loaddata initial_data.json, I got:

django.db.utils.IntegrityError: Problem installing fixture '/home/user/reponame/projectname/initial_data.json': Could not load contenttypes.ContentType(pk=2): UNIQUE constraint failed: django_content_type.app_label, django_content_type.model

What I did to make it work was just rename the pk to id for the contenttypes.contenttype model. After that, the migration worked as expected.

./manage.py loaddata initial_data.json 
Installed 2 object(s) from 1 fixture(s)

After the change, my initial_data.json file was:

[
{
    "model": "contenttypes.contenttype",
    "id": 1,
    "fields": {
        "app_label": "admin",
        "model": "logentry"
    }
},
{
    "model": "contenttypes.contenttype",
    "id": 2,
    "fields": {
        "app_label": "auth",
        "model": "permission"
    }
}]

It is worth mentioning that my original initial_dataj.json has many other models, but renaming pk to id only for the contenttypes.contenttype solved my problem.

Orly

Thank you, this works for me. I previously tried to change all pk to id, but that causes a django.contrib.contenttypes.models.DoesNotExist: ContentType matching query does not exist. KeyError: 'content_type' – Trichinosis

your solution is working perfectly. But what is the reason behind this? what are cons with this solution.? – Phocis

Hi @khadimhusen, I got to this point just by trial and error. – Orly

Same here, this solution works. Why don't they fix this? – Housel

I had to change "pk" to "id" in both contenttypes.contenttype and auth.permission. Then all fixtures loaded. – Macri

Answer 3

I had a similar problem. Inspired by this post:

https://github.com/yourlabs/django-cities-light/issues/89 (See 'how to fix it')

before running the loaddata command, I commented the receiver signal decorator before the 'saving function', and it worked.

Caiman

I cannot image a situation where you would want those post save hooks to run when doing a load of a db dump... – Rotund

Answer 4

2023 here and I stumbled on this and discovered why this can be happening. I am piggybacking off multiple answers here and it is not a comment because of what I need to explain.

Django, by default, has a signal that runs on every new model addition. When you add a new model to the application and run migrations, Django creates an entry on the ContentType model the moment a new instance of the model is added. This means that when you run dumpdata and then run loaddata, for each model, once the first record of each model is added to the DB, Django immediately creates a ContentType with FK to this model.

All is fine up until it gets to loading the ContentType in your json. At this point, the currently existing ContentTypes in the DB (which Django created via signals) clashes with the records in your JSON file.

The solution as others have pointed is to exclude ContentType from the dumpdata command. I will advise you also exclude everything from admin app which is basically log entries.

Next, you have to comment out any signals you may have before you run loaddata. If you don't do this, for every signal that exists, when loaddata loads a model that has a signal, this signal will run and create records in the DB which will inevitably cause failing constraints in for data in the json file that hasn't been created.

With this being said, your final command should be:

python manage.py dumpdata --natural-foreign --natural-primary -e contenttypes -e admin -e auth.Permission --indent
 2 > dump.json

And in your Django app, you comment out all code that involves signals. Then run:

python manage.py loaddata dump.json

When you're done (hopefully it installs successfully), you can uncomment your signal code and be happy!

I hope this works.

Hatchel

Answer 5

go the the json file and change evey single 'pk' to 'id' if you use vs code you can just select 1 and press cmd/ctrl + f2 as a shortcut

Mn

does not work for me as it causes the content_type error – Trichinosis

For me worked perfect from *.sqlite3 to MySQL! Thanks @Zeyad !, my error was: Could not load contenttypes.ContentType(pk=3): (1062, "Duplicate entry 'admin-logentry' for key 'django_content_type.django_content_type_app_label_mod – Abominate

Answer 6

Had the same problem while exporting and importing a model with a ManyToMany relation. This was, because I manually speficied the ManyToMany through model while exporting, what caused an unique constraint error.

class MyModel(model.Model):
    
    groups = models.ManyToManyField(
        'myapp.mymodel',        
        
    )

You only need to execute dumpdata myapp.mymodel, not `dumpdata myapp.mymodel myapp.mymodel_groups"

Otherwise, your through model data is in the export twice, and causes a unique constraint error.

It`s a good question how this behaves, when you specify an explicit through model...i don't know and have no time to test :)

Flemish

Answer 7

Comment your signals while loading the fixture.

Braille