Large ManyToMany relations on django admin form

Question

So, I have the following models:

class Band(models.Model):
    name = models.CharField(max_length=50)

class Contract(models.Model):
    band = models.ForeignKey(Band)
    when = models.DateTimeField(auto_now_add=True)
    salary = models.IntegerField()

class Musician(models.Model):
    first_name = models.CharField(max_length=50)
    last_name = models.CharField(max_length=50)
    instrument = models.CharField(max_length=100)
    bands = models.ManyToManyField(Band, through=Contract)

class Album(models.Model):
    artist = models.ForeignKey(Musician)
    name = models.CharField(max_length=100)
    release_date = models.DateField()
    num_stars = models.IntegerField()

So, I wanted to expose that on the admin page. So far, so good.

Note that our musicians here keep jumping in and out from bands. Some say one of them even had been on over 2 millions bands in his life-time. I don't know, maybe the bands are Whitesnake, Metallica or something.

How should we do that on the Django Admin Page?

I tried using raw_id_fields and apart the fact I didn't like the effect, it didn't work so well. It took a lot of time to load and it didn't let me add more ids. Weird.

I've used admin.StackedInline with no luck cause it will try to load every contract in a which, well, it's gonna take only 2 thousand years.

When Musician had a direct relation to Band it worked just fine with this library. But now that the relation isn't an straight one. Looks like autocomplete doesn't support it(it was getting slow anyway).

So, with all of this, I ask you lord SO members. What's the best way to do this? Is it autocomplete? Someone must have had to come across this issue!

Thanks in advance.

Burgett

Not sure how it performs with tons of records, but have you tried filter_horzontal or filter_vertical? – Edp

How would that work with many to many relations that have a through attribute? – Burgett

Some possible solutions here: #10111106 – Edp

Answer 1

To avoid loading every bands in your admin page use autocomplete_fields Django doc.
Just use it like that in your admin.py.

autocomplete_fields = ('bands',)

Then no bands will be pulled from DB to front, but you will be able to select it through a Select2 search field and it will be printed as "tags".

Shirring

This doesn't work as well as advertised :) The autocomplete widget will load all entries and populate the entire list at admin-page load (just tried this). It only works fine in this situation if you have an extremely sparse relation, like a few entries are lit up. It won't work if you have (like I had) thousands of relations lit up, it will give you a miles tall web page... – Uncover

OK, that's strange as we currently have thousands relations and we use autocomplete_fields and raw_id_fields without any issue. It's a life saver for us. – Shirring

I tested it by adding 5000 users to a many2many field, and set it as both read only or autocomplete and in both cases the django admin page tried to show all 5000 at the same time (in our production env we have 100000 lit up entries so there it would go complete crazy). I'd wish there was a way to get it to paginate (like autocomplete_fields normally does when it fetches the unlit entries). – Uncover

In addition to setting the autocomplete_fields, you have to also set search_fields on the ModelAdmin for the model whose values are being searched. – Goldstone

Answer 2

I found this solution and hope it will help somebody in the same situation:

I have many to many relations between the Product and Characteristic model. So, in the admin.py I am setting a form for a Product like the following where catch/get all the Characteristics and make the "prefecth_related" for Characteristic, as well the "select_related" could be done there:

class ProductAdminForm(forms.ModelForm):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.fields['characteristics'].queryset = Characteristic.objects.prefetch_related('category').all()

Brezhnev