How to override queryset count() method in Django's admin list

Question

In order to avoid time consuming and costly exact database count queries, I'd like to override the count() method inside a Django admin class like so:

from django.contrib import admin
from django.db import connection

class CountProxy:
    def __call__(self):
        # how to access the queryset `query` here?
        query = ...

        try:
            if not query.where:
                cursor = connection.cursor()
                cursor.execute("SELECT reltuples FROM pg_class WHERE relname = %s", [query.model._meta.db_table])
                n = int(cursor.fetchone()[0])
                if n >= 1000: return n # exact count for small tables
            return object_list.count()
        except:
            # exception for lists
            return len(object_list)
        return estimated_count

class MyAdmin(admin.ModelAdmin):
    def get_queryset(self, request):
        qs = super(MyAdmin, self).get_queryset(request)
        qs.count = CountProxy()
        return qs

But I don#t know how to access the original queryset within my CountProxy class. Any idea? I know I can overwrite the whole changelist view through get_changelist. But that involves a lot of duplicating code from Django's repo.

Broadcasting

object_list is not defined in your solution. return estimated_count is unreachable so will never execute, plus also undefined. – Martino

Answer 1

I could be wrong, but could you pass qs as an instance attribute for CountProxy?

class CountProxy:
    def __init__(self, query):
        self.query = query

    def __call__(self):
        # you've already had the query here, do something with self.query

class MyAdmin(admin.ModelAdmin):
    def get_queryset(self, request):
        qs = super(MyAdmin, self).get_queryset(request)
        qs.count = CountProxy(qs)
        return qs

Wirewove

Answer 2

This is what worked for me with postgres and Django 2.2.x

from django.db.models.query import QuerySet
from django.db import connection

class FastCountQuerySet(QuerySet):
    """
    Fast count (estimate) queryset to speedup count
    """
    def count(self):
        """
        Override count queries (performed by Django ORM) to display approximate value.
        This will speed up count i.e. in the admin interface.
        """
        if self._result_cache is not None:
            return len(self._result_cache)

        query = self.query
        if not (query.group_by or query.where or query.distinct):
            # cursor = connections[self.db].cursor()
            cursor = connection.cursor()
            cursor.execute("SELECT reltuples FROM pg_class WHERE relname = %s", [self.query.model._meta.db_table])
            n = int(cursor.fetchone()[0])
            if n >= 1000:
                return n  # exact count for small tables
            else:
                return self.query.get_count(using=self.db)
        else:
            return self.query.get_count(using=self.db)


class CustomManager(models.Manager):
    """
    Custom db manager
    """
    def get_queryset(self):
        return FastCountQuerySet(self.model)

And finally override your model manager:

class YourModel(models.Model):
    objects = CustomManager()

Martino

Answer 3

I did something similar before so I can help.

I defined a custom queryset class:

class MyQuerySet(QuerySet):

    def count(self):
        """
        Override count queries (performed by Django ORM) to display approximate value.
        This will speed the admin interface.

        """
        if self._result_cache is not None and not self._iter:
            return len(self._result_cache)

        query = self.query
        if not (query.group_by or query.having or query.distinct):
            cursor = connections[self.db].cursor()
            cursor.execute("SHOW TABLE STATUS LIKE '%s';" % self.model._meta.db_table)
            return cursor.fetchall()[0][4]
        else:
            return self.query.get_count(using=self.db)

Then defined a custom model manager:

class MyManager(models.Manager):

    def get_query_set(self):
        return MyQuerySet(self.model)

Then used it in my model:

class MyModel(models.Model):
    objects = MyManager()

Flavorous

Good solution, but I cannot override the Model's object manager. I need to apply this special count method only within Django's admin interface. – Broadcasting

The code in this answer doesn't really make sense as it is. What is return qs._clone(klass=ApproxCountQuerySet) supposed to do? The klass attribute is probably something from an external and non-visible code ... But this question/answer is actually the original of my own code/problem. – Broadcasting

Upvoted your suggestion, because it may certainly be a solution for others. – Broadcasting

Answer 4

If you're trying to make the admin page faster for large counts you can define a wrapper of models.Admin that as queryset returns a queryset that instead of the normal count uses an approximate one only when results ain't filetered. This would work for Postgres 12 and Django 4.1.7:

class FastCountAdmin(admin.ModelAdmin):
    class FastCountQuerySet(QuerySet):
        def count(self):
            """
            Override count queries (performed by Django ORM) to display approximate value.
            This will speed up count in the admin interface.
            """

            if self._result_cache is not None:
                return len(self._result_cache)

            query = self.query
            if not (query.group_by or query.where or query.distinct):
                cursor = connection.cursor()
                cursor.execute("SELECT reltuples FROM pg_class WHERE relname = %s", 
                    [self.model._meta.db_table])
                return int(cursor.fetchone()[0])
            else:
                return self.query.get_count(using=self.db)

    def get_queryset(self, request):
        qs = super().get_queryset(request)
        return FastCountAdmin.FastCountQuerySet(qs.model, using=qs.db)

Then you can use it like that:

class MyHugeModelAdmin(FastCountAdmin):
    model = MyHugeModel

Natascha