How to traverse a GenericForeignKey in Django?

Question

I'm using Django v1.9.4 with PostgreSQL 9.2.14 behind. With the following models:

from django.db import models
from django.contrib.contenttypes.fields import GenericRelation, GenericForeignKey
from django.contrib.contenttypes.models import ContentType

class Foo(models.Model):
    content_type = models.ForeignKey(ContentType)
    object_id = models.PositiveIntegerField()
    bar = GenericForeignKey('content_type', 'object_id')

class Bar(models.Model):
    foos = GenericRelation(Foo, related_query_name='bars')
    class Meta:
        abstract = True

class BarX(Bar):
    name = models.CharField(max_length=10, default='bar x')

class BarY(Bar):
    name = models.CharField(max_length=10, default='bar y')

Create some instances to demonstrate my problem:

>>> bar_x = BarX.objects.create()
>>> bar_y = BarY.objects.create()
>>> foo1 = Foo.objects.create(bar=bar_x)
>>> foo2 = Foo.objects.create(bar=bar_y)
>>> foo1.bar.name
u'bar x'
>>> foo2.bar.name
u'bar y'

I can't traverse the GFK in django, attempting to filter raises an exception with a message suggesting to add the GenericRelation. But using the generic relation, through related query name bars, is not working reliably. For example:

>>> [foo.bar.name for foo in Foo.objects.all()]
[u'bar x', u'bar y']  # in a pure python loop, it's working
>>> Foo.objects.filter(bar__name='bar x')
FieldError: Field 'bar' does not generate an automatic reverse relation and therefore cannot be used for reverse querying. If it is a GenericForeignKey, consider adding a GenericRelation.
>>> Foo.objects.values_list('bars__name', flat=1)
[None, u'bar y']   # but why None is returned in here?
>>> Foo.objects.filter(bars__name='bar x')
[]  # why no result here?
>>> Foo.objects.filter(bars__name='bar y')
[<Foo: Foo object>]  # but this one works?

What am I doing wrong?

---

Cautionary note to future readers: Templating related_query_name on GenericRelation doesn't work properly on Django 1.9.

Added in Django 1.10 was related_query_name now supports app label and class interpolation using the '%(app_label)s' and '%(class)s' strings, after the fix for #25354 was merged.

If you're on Django 1.10, you can go ahead and put the GenericRelation on the abstract base class and template it like related_query_name='%(app_label)s_%(class)s' to ensure uniqueness across the subclasses.

Shading

Answer 1

In general, it is not possible to traverse a GenericForeignKey in this direction in the way you are trying. A GenericForeignKey can point to any model in your application at all, not only Bar and its subclasses. For that reason, Foo.objects.filter(bar__somefield='some value') cannot know what target model you have in mind at the moment, and therefore it is impossible to tell what fields that target model has. In fact, there is no way to pick what database table to join with when performing such a query – it could be any table, depending on the value of Foo.content_type.

If you do want to use a generic relation in joins, you'll have to define a GenericRelation on the other end of that relationship. That way you can let Django know which model it is supposed to look for on the other side.

For instance, you can create your BarX and BarY models like this:

class BarX(Bar):
    name = models.CharField(max_length=10, default='bar x')
    foos = GenericRelation(Foo, related_query_name='bar_x')

class BarY(Bar):
    name = models.CharField(max_length=10, default='bar y')
    foos = GenericRelation(Foo, related_query_name='bar_y')

If you do this, then you can perform queries like the following:

Foo.objects.filter(bar_x__name='bar x')
Foo.objects.filter(bar_y__name='bar y')

However, you have to pick a single target model. That is a limitation that you cannot really overcome in any way; every database join needs to know in advance which tables it operates on.

If you absolutely need to allow both BarX and BarY as the target, you should be able to list both of them explicitly in your query filter using a Q expression:

Foo.objects.filter(Q(bar_x__name='bar x') | Q(bar_y__name='bar y'))

Swatter

OK so it seems to be a limitation due to the sql join. But for the examples in my question, why does django permit the queries at all? Shouldn't it rather raise an exception , than return incorrect results? – Shading

Yeah, that looks like a bug, I don't think it should allow you to make such queries. Could you perhaps add the SQL generated by those queries to your question? – Swatter

I played with your solution a bit and found another interesting thing, it may not be necessary to put the GenericRelation field on each and every subclass with a unique related_query_name. You can actually leave it on the base class and template it like related_query_name='%(app_label)s_%(class)s' to ensure uniqueness. – Shading

Yeah, I thought that might be the case, but I wasn't sure it worked for GenericRelations, too, so I chose not to mention it... Glad to see it does work. – Swatter