How do I query Wagtail tags without it creating hundreds of queries, n+1 problem?

Question

I have a handful of entries that each has dozens of "tags" when I try to fetch that data it causes hundreds of queries. While using prefetching works on all of my other models and links, it does not seem to work on this. The post below says it has something related to wagtails fake query function but it hasn't been fixed yet. Is there a way to utilize tags where I can fetch them via the api so they don't create hundreds of queries (one for each tag) making them unusable due to their slow response time?

Any help you can provide would be much appreciated.

A similar issue was brought up here but got no answer How to prefetch Wagtail post tags?

Also, when I use prefetch it actually doubles the number of queries

Here is my code:

# serializer.py////////////////////////////////////

# orderable models links
# Custom searializers to add custom display fields
class EventCollectionSerializer(serializers.ModelSerializer):
    class Meta:
         model = CategoryEventCollection
         fields = ['id', 'collection_name', 'collection_event']

class TagSerializer(serializers.ModelSerializer):
    class Meta:
         model = Tag
         fields = ['name', 'id']


# MAIN SERIALIZER PRIMARY FOCUS
class PrimaryFocusSerializer(serializers.HyperlinkedModelSerializer):

     # assign fields to custom serializer
     event_collection = EventCollectionSerializer(many=False)
     tags = TagSerializer(many=True)


class Meta:
    model = PrimaryFocus
    fields = ['id',
              'event_collection',
              'title_major',
              'synopsis',
              'level_ability',
              'page_count',
              'reading_category',
              'book_image',
              'author_first_name',
              'author_last_name',
              'author_dob',
              'tags',
              'purchase_link',
              'source',
              ]



# PRIMARY FOCUS VIEWSET//////////////////////////////////
class PrimaryFocusViewSet(viewsets.ModelViewSet):

    # This snippet just console logs how many queries
    def dispatch(self, *args, **kwargs):
        response = super().dispatch(*args, **kwargs)
        print('Queries Counted: {}'.format(len(connection.queries)))
        return response

# fetch all lexis items to setup queryset
queryset = PrimaryFocus.objects.all()

# set serializer class
serializer_class = PrimaryFocusSerializer

# optimize query with select related and prefetching
def get_queryset(self):
    return (PrimaryFocus.objects
            .select_related('event_collection',)
            .prefetch_related('tags', )
            .all()
            )

Dahlia

have you tried using custom serializer instead of a ModelSerializer for tags and see if it makes any changes? – Nightshade

One of these projects might be of use: github.com/tolomea/django-auto-prefetch - or - github.com/GeeWee/django-auto-prefetching – Normi

@mahyar, do you mean to create a custom serializer that doesn't inherit from ModelSerializer for TagSerializer? I am unfamiliar with how to do that from scratch, do you have any links you can direct me towards to go through that? What benefits will that potentially provide? Wagtail change log from 2016 says * Fix: prefetch_related on a ClusterTaggableManager no longer fails (but doesn't prefetch either)* In wagtail I am using the ClusterTaggableManager to create tags so would a custom serializer overcome this limitation? – Dahlia

@DanSwain Yes I have used Django-auto-prefetching and while it works on all of my other models and serializers it does not seem to work on Tags. Any idea on how to overcome this? – Dahlia