Nested annotate fields in Django REST Framework serializers
Asked Answered
I

3

6

I am trying to view nested annotate (aggregated/calculated) fields in Django REST Framework serializers. This would allow to work more cleanly with annotated fields. This post is similar to Aggregate (and other annotated) fields in Django Rest Framework serializers however I would like a similar technique to work nested. Below the methodology is visible on how this works without nesting and how it doesn't seem to work with nesting.

I know this could be achieved manually (with a Django View) or by using methods that overload the database which I am not interested in. But maybe there is a performant and elegant solution for this problem.

The following works (not nested)

Models

class IceCreamCompany(models.Model):
    name = models.CharField(max_length=255)


class IceCreamTruck(models.Model):
    company = models.ForeignKey('IceCreamCompany', related_name='trucks')
    capacity = models.IntegerField()


class IceCreamTruckDriver(models.Model):
    name = models.CharField(max_length=255)
    first_name = models.CharField(max_length=255)
    truck = models.ForeignKey('IceCreamTruck', related_name='drivers')

Serializers

class IceCreamTruckDriverSerializer(serializers.ModelSerializer):

    class Meta:
        model = IceCreamTruckDriver
        fields = ('name', 'first_name')


class IceCreamTruckSerializer(serializers.ModelSerializer):
    drivers = IceCreamTruckDriverSerializer(many=True, read_only=True)

    class Meta:
        model = IceCreamTruck
        fields = ('capacity', 'drivers')


class IceCreamCompanySerializer(serializers.ModelSerializer):
    trucks = IceCreamTruckSerializer(many=True, read_only=True)
    amount_of_trucks = serializers.IntegerField()

    class Meta:
        model = IceCreamCompany
        fields = ('name', 'trucks', 'amount_of_trucks')

Viewset

class IceCreamCompanyViewSet(viewsets.ModelViewSet):
    queryset = IceCreamCompany.objects.prefetch_related('trucks', 'trucks__drivers')\
                           .annotate(amount_of_trucks=Count('trucks'))\
                           .all()

    serializer_class = IceCreamCompanySerializer

Result

"results": [
        {
            "name": "Pete Ice Cream",
            "trucks": [
                {
                    "capacity": 35,
                    "drivers": [
                        {
                            "name": "Damian",
                            "first_name": "Ashley"
                        },
                        {
                            "name": "Wilfrid",
                            "first_name": "Lesley"
                        }
                    ]
                },
                {
                    "capacity": 30,
                    "drivers": [
                        {
                            "name": "Stevens",
                            "first_name": "Joseph"
                        }
                    ]
                },
                {
                    "capacity": 30,
                    "drivers": []
                }
            ],
            "amount_of_trucks": 3
        }
    ]

The following does not work (nested)

Same models

Serializers

class IceCreamTruckDriverSerializer(serializers.ModelSerializer):

    class Meta:
        model = IceCreamTruckDriver
        fields = ('name', 'first_name')


class IceCreamTruckSerializer(serializers.ModelSerializer):
    drivers = IceCreamTruckDriverSerializer(many=True, read_only=True)
    amount_of_drivers = serializers.IntegerField()

    class Meta:
        model = IceCreamTruck
        fields = ('capacity', 'drivers', 'amount_of_drivers')


class IceCreamCompanySerializer(serializers.ModelSerializer):
    trucks = IceCreamTruckSerializer(many=True, read_only=True)

    class Meta:
        model = IceCreamCompany
        fields = ('name', 'trucks')

Viewset

class IceCreamCompanyViewSet(viewsets.ModelViewSet):
    queryset = IceCreamCompany.objects.prefetch_related('trucks', 'trucks__drivers')\
                           .annotate(trucks__amount_of_drivers=Count('trucks__drivers'))\
                           .all()

    serializer_class = IceCreamCompanySerializer

Result

AttributeError at /ice/
Got AttributeError when attempting to get a value for field `amount_of_drivers` on serializer `IceCreamTruckSerializer`.
The serializer field might be named incorrectly and not match any attribute or key on the `IceCreamTruck` instance.
Original exception text was: 'IceCreamTruck' object has no attribute 'amount_of_drivers'.
Inquire answered 6/10, 2015 at 8:9 Comment(5)
Shouldn't you name the field like amount_of_drivers instead of trucks__amount_of_drivers ?Philosopher
If i use amount_of_drivers in the annotate function in the queryset it tries to find the amount_of_drivers field in the IceCreamCompanySerializer, which would not be a nested annotate field. I want it to be possible inside the IceCreamTruckSerializer.Inquire
No sir, what I'm merelly suggesting is to rename the annotation: .annotate(amount_of_drivers=Count('trucks__drivers')); the error names this issue, the fact that the name of the annotated column hasn't been found in the queryset. The name of the annotated column is simply a name, doesn't traverse relations for you.Philosopher
Hi, were you able to achieve this in the end without using the SerializerMethodField?Luigiluigino
@NicholasColes Yes, I do still use a custom View, which is not optimal. You could however try out a custom manager, as stated by Campi in the other answer. For my use-case that approach would make it too complicated. I haven't checked the amount of queries either.Inquire
I
1

I got an answer using the Django REST google groups to use read_only=True inside the IntegerField, which helped removing the error but then the field wasn't displayed anymore. Maybe my annotation was wrong. Anyway I ended up using a custom view in Django since I ended up needing more data. However you can get the data in other ways:

A very elegant solution would be to remove the annotate function and use a SerializerMethodField which can give me my result.

HOWEVER: this does make a lot of queries!!

Same models

Serializers

class IceCreamTruckDriverSerializer(serializers.ModelSerializer):

    class Meta:
        model = IceCreamTruckDriver
        fields = ('name', 'first_name')


class IceCreamTruckSerializer(serializers.ModelSerializer):
    drivers = IceCreamTruckDriverSerializer(many=True, read_only=True)
    amount_of_drivers = serializers.SerializerMethodField()

    def get_amount_of_drivers(self, obj):
        return obj.drivers.count()

    class Meta:
        model = IceCreamTruck
        fields = ('capacity', 'drivers', 'amount_of_drivers')


class IceCreamCompanySerializer(serializers.ModelSerializer):
    trucks = IceCreamTruckSerializer(many=True, read_only=True)

    class Meta:
        model = IceCreamCompany
        fields = ('name', 'trucks')

Viewset

class IceCreamCompanyViewSet(viewsets.ModelViewSet):
    queryset = IceCreamCompany.objects.prefetch_related('trucks', 'trucks__drivers').all()

    serializer_class = IceCreamCompanySerializer

Result

"results": [
        {
            "name": "Pete Ice Cream",
            "trucks": [
                {
                    "capacity": 35,
                    "drivers": [
                        {
                            "name": "Damian",
                            "first_name": "Ashley"
                        },
                        {
                            "name": "Wilfrid",
                            "first_name": "Lesley"
                        }
                    ],
                    "amount_of_drivers": 2
                },
                {
                    "capacity": 30,
                    "drivers": [
                        {
                            "name": "Stevens",
                            "first_name": "Joseph"
                        }
                    ],
                    "amount_of_drivers": 1
                },
                {
                    "capacity": 30,
                    "drivers": [],
                    "amount_of_drivers": 0
                }
            ]
        }
    ]

It's also possible to use functions inside the models like this: Django Rest Framework Ordering on a SerializerMethodField (it's visible in the code itself) but I didn't choose it so I don't have to modify my models too much. This also makes too many queries.

Inquire answered 15/10, 2015 at 12:6 Comment(0)
S
12

For reference, it is also possible to annotate the amount of drivers per Truck on the model IceCreamTruck, for example with a custom manager:

class AnnotatedManager(models.Manager):
    def get_queryset(self):
        return super().get_queryset().annotate(amount_of_drivers=Count('drivers'))

class IceCreamTruck(models.Model):
    company = models.ForeignKey('IceCreamCompany', related_name='trucks')
    capacity = models.IntegerField()

    objects = AnnotatedManager()

Then you don't need to annotate the viewset because amount_of_drivers is already annotated on trucks:

class IceCreamCompanyViewSet(viewsets.ModelViewSet):
    queryset = IceCreamCompany.objects.prefetch_related('trucks', 'trucks__drivers').all()
    serializer_class = IceCreamCompanySerializer 

It should be more efficient than counting inside the serializer.

Scabbard answered 8/4, 2018 at 9:43 Comment(2)
Good idea, and so is "number of drivers." ;-)Palmitin
Thir is the only way how to add annotated fields to nested serializers.Roguery
I
1

I got an answer using the Django REST google groups to use read_only=True inside the IntegerField, which helped removing the error but then the field wasn't displayed anymore. Maybe my annotation was wrong. Anyway I ended up using a custom view in Django since I ended up needing more data. However you can get the data in other ways:

A very elegant solution would be to remove the annotate function and use a SerializerMethodField which can give me my result.

HOWEVER: this does make a lot of queries!!

Same models

Serializers

class IceCreamTruckDriverSerializer(serializers.ModelSerializer):

    class Meta:
        model = IceCreamTruckDriver
        fields = ('name', 'first_name')


class IceCreamTruckSerializer(serializers.ModelSerializer):
    drivers = IceCreamTruckDriverSerializer(many=True, read_only=True)
    amount_of_drivers = serializers.SerializerMethodField()

    def get_amount_of_drivers(self, obj):
        return obj.drivers.count()

    class Meta:
        model = IceCreamTruck
        fields = ('capacity', 'drivers', 'amount_of_drivers')


class IceCreamCompanySerializer(serializers.ModelSerializer):
    trucks = IceCreamTruckSerializer(many=True, read_only=True)

    class Meta:
        model = IceCreamCompany
        fields = ('name', 'trucks')

Viewset

class IceCreamCompanyViewSet(viewsets.ModelViewSet):
    queryset = IceCreamCompany.objects.prefetch_related('trucks', 'trucks__drivers').all()

    serializer_class = IceCreamCompanySerializer

Result

"results": [
        {
            "name": "Pete Ice Cream",
            "trucks": [
                {
                    "capacity": 35,
                    "drivers": [
                        {
                            "name": "Damian",
                            "first_name": "Ashley"
                        },
                        {
                            "name": "Wilfrid",
                            "first_name": "Lesley"
                        }
                    ],
                    "amount_of_drivers": 2
                },
                {
                    "capacity": 30,
                    "drivers": [
                        {
                            "name": "Stevens",
                            "first_name": "Joseph"
                        }
                    ],
                    "amount_of_drivers": 1
                },
                {
                    "capacity": 30,
                    "drivers": [],
                    "amount_of_drivers": 0
                }
            ]
        }
    ]

It's also possible to use functions inside the models like this: Django Rest Framework Ordering on a SerializerMethodField (it's visible in the code itself) but I didn't choose it so I don't have to modify my models too much. This also makes too many queries.

Inquire answered 15/10, 2015 at 12:6 Comment(0)
B
0

In case you don't want to override the manager of the model (as suggested by Campi), since it would have impact on all queries from the given model, there is one more way to achieve this.

  1. Define a function in IceCreamCompany model where the counting of drivers is done along with the prefetching of related objects

    class IceCreamCompany(models.Model):
        name = models.CharField(max_length=255)
    
        def get_annotated_trucks(self):
            return IceCreamTruck.objects.filter(company_id=self.id).annotate(
            amount_of_drivers=Count('drivers')).prefetch_related('drivers')
    
  2. Use the function to fetch IceCreamTruck objects in IceCreamCompanySerializer by specifying the function in the source parameter of the nested serializer

    class IceCreamCompanySerializer(serializers.ModelSerializer):
        trucks = IceCreamTruckSerializer(many=True, read_only=True, 
        source='get_annotated_trucks')
    
        class Meta:
            model = IceCreamCompany
            fields = ('name', 'trucks')
    
  3. There is no need to do any action in the viewset (prefetching/annotating)

    class IceCreamCompanyViewSet(viewsets.ModelViewSet):
        queryset = IceCreamCompany.objects.all()
        serializer_class = IceCreamCompanySerializer
    

Note: this approach produces the same queries as the approach with custom manager

Bloodshed answered 30/12, 2023 at 10:5 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.