Annotations are ignored in Django/Tastypie

Question

In my Tastypie resource, I'm annotating my queryset, and yet I don't see that annotation flow through to the JSON Tastypie generates and passes back. The code is straightforward:

class CompetitionResource(ModelResource):
    total_tickets = fields.IntegerField(readonly=True)

    class Meta:
        queryset = Competition.objects.all().annotate(total_tickets=Count('ticket__ticketownership__user__id', distinct=True))

That Count I'm generating and annotating there in my queryset simply does not show up in the final JSON. The final JSON has a total_users field (because I declared one in my ModelResource), but it's null. Am I missing anything obvious to make sure annotations like this get passed through? If not, what would be a way to solve this?

One way to do it is to create an attribute in my Model and then tie the total_users field in my ModelResource to that attribute. But that would presumably result in a Count query for each individual Competition I pull from the database, and that is not good. I want to do it in one annotation-type query.

Precious

I haven't seen annotate() used with all() before. What happens if you just use Competition.objects.annotate(total_tickets=Count('ticket__ticketownership__user__id', distinct=True))// edit: I just looked at the Django code, and these calls should do the same thing...sorry. – Lacerated

Yeah, that shouldn't make a difference. The annotate() code above works fine, it's just that it doesn't get passed through to the eventual JSON output. – Precious

Answer 1

Ok, I got it. You can simply use the custom dehydrate_[field name] methods that you can add to a ModelResource. For each ModelResource field, Tastypie checks if you specified a dehydrate_[field name] method, and if you did, then it calls that method when it processes an object into a bundle (which then gets put out as JSON or XML or whatever). This dehydrate_[field name] method gets the bundle that Tastypie has created up until that point, for that particular object. The good thing is that this bundle has the original object in it, under bundle.obj. And that object will still have the original annotation you provided in get_object_list (as shown in the answer above). So you can use the following code.

class CompetitionResource(ModelResource):
    total_tickets = fields.IntegerField(readonly=True)

    class Meta:
        queryset = Competition.objects.all()

    def get_object_list(self, request):
        return super(CompetitionResource, self).get_object_list(request).annotate(total_tickets=Count('ticket__ticketownership__user__id', distinct=True))

    def dehydrate_total_tickets(self, bundle):
        return bundle.obj.total_tickets

Whatever you return from the custom dehydrate_[field name] method will be properly stored as that field's final value in the bundle for that object, and then properly processed into output.

Precious

Answer 2

It doesn't come up in the docs, but looking at the source there is an attribute argument that can be passed to the field declaration that can be used to tie it to an attribute of the model instance.

Optionally accepts an attribute, which should be a string of either an instance attribute or callable off the object during the dehydrate or push data onto an object during the hydrate. Defaults to None, meaning data will be manually accessed.

So for your example, the following should do the trick.

class CompetitionResource(ModelResource):
    total_tickets = fields.IntegerField(readonly=True, attribute='total_tickets')

    class Meta:
        queryset = Competition.objects.all().annotate(total_tickets=Count('ticket__ticketownership__user__id', distinct=True))

The dehydrate solution works for populating the sent object with values, but doesn't allow you to easily take advantage of some of Tastypie's other features such as built-in filtering (and I believe sorting). Using a field definition with an attribute argument does.

Ballot

Answer 3

I think your problem may be related to the warning given in the Tastypie docs for queryset:

If you place any callables in this, they’ll only be evaluated once (when the Meta class is instantiated). This especially affects things that are date/time related. Please see the :ref:cookbook for a way around this.

Looking at the relevant section in the cookbook, I think you should try something like this:

class CompetitionResource(ModelResource):
    total_users = fields.IntegerField(readonly=True)

    class Meta:
        queryset = Competition.objects.all()

    def get_object_list(self, request):
        return super(CompetitionResource, self).get_object_list(request).annotate(total_tickets=Count('ticket__ticketownership__user__id', distinct=True))

Lacerated

Good thought - so good that in fact I had tried it before as well... but it doesn't work either. total_tickets is still null (slight bug in your code by the way, your field name is total_users and your annotation name is total_tickets, but it doesn't work anyway). I'll keep looking. Thanks a lot though. – Precious