In django, is there a way to directly annotate a query with a related object in single query?

Question

Consider this query:

query = Novel.objects.< ...some filtering... >.annotate(
    latest_chapter_id=Max("volume__chapter__id")
)

Actually what I need is to annotate each Novel with its latest Chapter object, so after this query, I have to execute another query to select actual objects by annotated IDs. IMO this is ugly. Is there a way to combine them into a single query?

Ratsbane

Could you annotate chapters with novels instead? – Pradeep

Seems not possible to me, because I need only the latest chapter for each novel, but a novel have many chapters, and chapters are in different volumes. As far as I know distinct() is useless in this case (correct me if I am wrong), and I don't know any other way to select exactly one chapter except for starting from novel. Any ideas? – Ratsbane

Answer 1

No, it's not possible to combine them into a single query.

You can read the following blog post to find two workarounds.

Threefold

The answer of rune-kaagaard is better now. – Purgatory

The linked post suggests using extra() in the query. As of Django 1.8, extra() is not recommended and to be deprecated. – Currajong

Now possible with Django 3.2 using JSONObject, see my answer. – Touslesmois

Answer 2

Yes, it's possible.

To get a queryset containing all Chapters which are the last in their Novels, simply do:

from django.db.models.expressions import F
from django.db.models.aggregates import Max

Chapters.objects.annotate(last_chapter_pk=Max('novel__chapter__pk')
    ).filter(pk=F('last_chapter_pk'))

Tested on Django 1.7.

Curricle

Answer 3

Possible with Django 3.2+

Make use of django.db.models.functions.JSONObject (added in Django 3.2) to combine multiple fields (in this example, I'm fetching the latest object, however it is possible to fetch any arbitrary object provided that you can get LIMIT 1) to yield your object):

MainModel.objects.annotate(
    last_object=RelatedModel.objects.filter(mainmodel=OuterRef("pk"))
    .order_by("-date_created")
    .values(
        data=JSONObject(
            id="id", body="body", date_created="date_created"
        )
    )[:1]
)

Invercargill

Oh, I must have totally missed JSONObject - I've just been using my own Func subclass and JsonBuildObject, but I think this is much neater...although sometimes my keys are field values too, rather than fixed keys. – Anthropomorphism

What if I want to later through django rest api sort or filter using this field? – Forethought

Answer 4

Yes, using Subqueries:

from django.db.models import OuterRef, Subquery

latest_chapters = (
    Chapter.objects
    .filter(novel=OuterRef("pk"))
    .order_by("chapter_order")
)

novels_with_chapter = Novel.objects.annotate(
    latest_chapter=Subquery(latest_chapters.values("chapter")[:1])
)

Tested on Django 3.0

The subquery creates a select statement inside the select statement for the novels, then adds this as an annotation. This means you only hit the database once.

I also prefer this to Rune's answer as it actually annotates a Novel object.

Hope this helps, anyone who came looking like much later like I did.

Detradetract

Unfortunately does not work for me as it seems subqueries can only return a single column and not a complete object. – Shakira

@Gnietschow, Yes if you want more than just one column you are probably better off using something like prefetch_related docs – Detradetract

Unfortunately that does not work for me as I want to sort and filter for fields of both models. – Shakira

@Shakira See my answer that yields multiple columns in a single subquery. – Touslesmois