can reduce multiple SELECT
queries to only 1 SELECT
query with Forward Foreign Key and Reverse Foreign Key in one-to-one relationship and with Forward Foreign Key in one-to-many and many-to-many relationships.
cannot be used with Reverse Foreign Key in one-to-many and many-to-many relationships.
- can reduce multiple
SELECT
queries to 2 SELECT
queries as a minimum with Forward Foreign Key and Reverse Foreign Key in one-to-one, one-to-many and many-to-many relationships.
*You can see my answer explaining the meaning of Forward Foreign Key and Reverse Foreign Key.
The following shows my experiments of select_related()
and prefetch_related()
with Forward Foreign Key and Reverse Foreign Key in one-to-one, one-to-many and many-to-many relationships.
<One-to-one relationship>
For example, there are Person
and PersonDetail
models which have one-to-one relationship as shown below:
# "app/models.py
from django.db import models
class Person(models.Model):
name = models.CharField(max_length=20)
def __str__(self):
return self.name
class PersonDetail(models.Model):
person = models.OneToOneField(Person, on_delete=models.CASCADE)
age = models.IntegerField()
gender = models.CharField(max_length=20)
def __str__(self):
return str(self.age) + " " + self.gender
And, there are Person
and PersonDetail
admins as shown below:
# "app/admin.py
from django.contrib import admin
from .models import Person, PersonDetail
class PersonDetailInline(admin.TabularInline):
model = PersonDetail
@admin.register(Person)
class PersonAdmin(admin.ModelAdmin):
inlines = (PersonDetailInline,)
list_display = ('id', 'name')
ordering = ('id',)
@admin.register(PersonDetail)
class PersonDetailAdmin(admin.ModelAdmin):
list_display = ('id', 'age', 'gender', 'person')
ordering = ('id',)
Then, Person
admin has 5 objects as shown below:
And, PersonDetail
admin has 5 objects as shown below:
<Forward Foreign Key>
Then, I iterate Person
model from PersonDetail
model as shown below:
for obj in PersonDetail.objects.all():
print(obj.person)
Then, these below are outputted on console:
John
David
Lisa
Kai
Anna
Then, 6 SELECT
queries are run as shown below. *I use PostgreSQL and these below are the query logs of PostgreSQL and you can see my answer explaining how to enable and disable the query logs on PostgreSQL:
Next, I iterate Person
model from PersonDetail
model with select_related("person")
as shown below. *The order of select_related()
and all()
don't matter:
for obj in PersonDetail.objects.select_related("person").all():
print(obj.person)
Then, these below are outputted on console:
John
David
Lisa
Kai
Anna
Then, 1 SELECT
query is run as shown below:
Next, I iterate Person
model from PersonDetail
model with prefetch_related("person")
as shown below. *The order of prefetch_related()
and all()
don't matter:
for obj in PersonDetail.objects.prefetch_related("person").all():
print(obj.person)
Then, these below are outputted on console:
John
David
Lisa
Kai
Anna
Then, 2 SELECT
queries are run as shown below:
<Reverse Foreign Key>
Next, I iterate PersonDetail
model from Person
model as shown below:
for obj in Person.objects.all():
print(obj.persondetail)
Then, these below are outputted on console:
32 Male
18 Male
26 Female
36 Male
21 Female
Then, 6 SELECT
queries are run as shown below:
Next, I iterate PersonDetail
model from Person
model with select_related("persondetail")
as shown below:
for obj in Person.objects.select_related("persondetail").all():
print(obj.persondetail)
Then, these below are outputted on console:
32 Male
18 Male
26 Female
36 Male
21 Female
Then, 1 SELECT
query is run as shown below:
Next, I iterate PersonDetail
model from Person
model with prefetch_related("persondetail")
as shown below:
for obj in Person.objects.prefetch_related("persondetail").all():
print(obj.persondetail)
Then, these below are outputted on console:
32 Male
18 Male
26 Female
36 Male
21 Female
Then, 2 SELECT
queries are run as shown below:
<One-to-many relationship>
For example, there are Category
and Product
models which have one-to-many relationship as shown below:
# "app/models.py"
from django.db import models
class Category(models.Model):
name = models.CharField(max_length=20)
def __str__(self):
return self.name
class Product(models.Model):
category = models.ForeignKey(Category, on_delete=models.CASCADE)
name = models.CharField(max_length=50)
price = models.DecimalField(decimal_places=2, max_digits=5)
def __str__(self):
return str(self.category) + " " + self.name + " " + str(self.price)
And, there are Category
and Product
admins as shown below:
# "app/admin.py"
from django.contrib import admin
from .models import Category, Product
@admin.register(Category)
class CategoryAdmin(admin.ModelAdmin):
list_display = ('id', 'name')
ordering = ('id',)
@admin.register(Product)
class ProductAdmin(admin.ModelAdmin):
list_display = ('id', 'name', 'price', 'category')
ordering = ('id',)
Then, Category
admin has 4 objects as shown below:
And, Product
admin has 6 objects as shown below:
<Forward Foreign Key>
Then, I iterate Category
model from Product
model as shown below:
for obj in Product.objects.all():
print(obj.category)
Then, these below are outputted on console:
Fruits
Fruits
Vegetable
Meat
Meat
Fish
Then, 7 SELECT
queries are run as shown below:
Next, I iterate Category
model from Product
model with select_related("category")
as shown below:
for obj in Product.objects.select_related("category").all():
print(obj.category)
Then, these below are outputted on console:
Fruits
Fruits
Vegetable
Meat
Meat
Fish
Then, 1 SELECT
query is run as shown below:
Next, I iterate Category
model from Product
model with prefetch_related("category")
as shown below:
for obj in Product.objects.prefetch_related("category").all():
print(obj.category)
Then, these below are outputted on console:
Fruits
Fruits
Vegetable
Meat
Meat
Fish
Then, 2 SELECT
queries are run as shown below:
<Reverse Foreign Key>
Next, I iterate Product
model from Category
model as shown below:
for obj in Category.objects.all():
print(obj.product_set.all())
Then, these below are outputted on console:
<QuerySet [<Product: Fruits Apple 10.00>, <Product: Fruits Orange 20.00>]>
<QuerySet [<Product: Vegetable Carrot 30.00>]>
<QuerySet [<Product: Meat Chicken 40.00>, <Product: Meat Beef 50.00>]>
<QuerySet [<Product: Fish Salmon 60.00>]>
Then, 5 SELECT
queries are run as shown below:
Next, I tries to iterate Product
model from Category
model with select_related("product_set")
as shown below:
for obj in Category.objects.select_related("product_set").all():
print(obj.product_set.all())
Then, the error below occurs because select_related("product_set")
cannot be used with Reverse Foreign Key:
django.core.exceptions.FieldError: Invalid field name(s) given in
select_related: 'product_set'. Choices are: (none)
Actually, there is no error if I use select_related()
with no argument as shown below:
# ↓ No argument
for obj in Category.objects.select_related().all():
print(obj.product_set.all())
Then, these below are outputted on console:
<QuerySet [<Product: Fruits Apple 10.00>, <Product: Fruits Orange 20.00>]>
<QuerySet [<Product: Vegetable Carrot 30.00>]>
<QuerySet [<Product: Meat Chicken 40.00>, <Product: Meat Beef 50.00>]>
<QuerySet [<Product: Fish Salmon 60.00>]>
But, 5 SELECT
queries are still run instead of 1 SELECT
query as shown below:
Next, I iterate Product
model from Category
model with prefetch_related("product_set")
as shown below:
for obj in Category.objects.prefetch_related("product_set").all():
print(obj.product_set.all())
Then, these below are outputted on console:
<QuerySet [<Product: Fruits Apple 10.00>, <Product: Fruits Orange 20.00>]>
<QuerySet [<Product: Vegetable Carrot 30.00>]>
<QuerySet [<Product: Meat Chicken 40.00>, <Product: Meat Beef 50.00>]>
<QuerySet [<Product: Fish Salmon 60.00>]>
Then, 2 SELECT
queries are run as shown below:
<Many-to-many relationship>
For example, Category
and Product
models have many-to-many relationship as shown below:
# "app/models.py"
from django.db import models
class Category(models.Model):
name = models.CharField(max_length=20)
def __str__(self):
return self.name
class Product(models.Model):
categories = models.ManyToManyField(Category)
name = models.CharField(max_length=50)
price = models.DecimalField(decimal_places=2, max_digits=5)
def __str__(self):
return self.name + " " + str(self.price)
And, there are Category
and Product
admins as shown below:
# "app/admin.py
from django.contrib import admin
from .models import Category, Product
@admin.register(Category)
class CategoryAdmin(admin.ModelAdmin):
list_display = ('id', 'name', 'get_products')
ordering = ('id',)
@admin.display(description='products')
def get_products(self, obj):
return [product.name for product in obj.product_set.all()]
@admin.register(Product)
class ProductAdmin(admin.ModelAdmin):
list_display = ('id', 'name', 'price', 'get_categories')
ordering = ('id',)
@admin.display(description='categories')
def get_categories(self, obj):
return [category.name for category in obj.categories.all()]
Then, Category
admin has 5 objects as shown below:
And, Product
admin has 6 objects as shown below:
<Forward Foreign Key>
Then, I iterate Category
model from Product
model as shown below:
for obj in Product.objects.all():
print(obj.categories.all())
Then, these below are outputted on console:
<QuerySet [<Category: Fruits>, <Category: 20% OFF>]>
<QuerySet [<Category: Fruits>]>
<QuerySet [<Category: Vegetable>]>
<QuerySet [<Category: Meat>, <Category: 20% OFF>]>
<QuerySet [<Category: Meat>]>
<QuerySet [<Category: Fish>, <Category: 20% OFF>]>
Then, 7 SELECT
queries are run as shown below:
Next, I iterate Category
model from Product
model with select_related("categories")
as shown below:
for obj in Product.objects.select_related("categories").all():
print(obj.categories.all())
Then, the error below occurs because select_related("categories")
cannot be used with Reverse Foreign Key:
django.core.exceptions.FieldError: Invalid field name(s) given in
select_related: 'categories'. Choices are: (none)
Actually, there is no error if I use select_related()
with no argument as shown below:
# ↓ No argument
for obj in Product.objects.select_related().all():
print(obj.categories.all())
Then, these below are outputted on console:
<QuerySet [<Category: Fruits>, <Category: 20% OFF>]>
<QuerySet [<Category: Fruits>]>
<QuerySet [<Category: Vegetable>]>
<QuerySet [<Category: Meat>, <Category: 20% OFF>]>
<QuerySet [<Category: Meat>]>
<QuerySet [<Category: Fish>, <Category: 20% OFF>]>
But, 7 SELECT
queries are still run instead of 1 SELECT
query as shown below:
Next, I iterate Category
model from Product
model with prefetch_related("categories")
as shown below:
for obj in Product.objects.prefetch_related("categories").all():
print(obj.categories.all())
Then, these below are outputted on console:
<QuerySet [<Category: Fruits>, <Category: 20% OFF>]>
<QuerySet [<Category: Fruits>]>
<QuerySet [<Category: Vegetable>]>
<QuerySet [<Category: Meat>, <Category: 20% OFF>]>
<QuerySet [<Category: Meat>]>
<QuerySet [<Category: Fish>, <Category: 20% OFF>]>
Then, 2 SELECT
queries are run as shown below:
<Reverse Foreign Key>
Next, I iterate Product
model from Category
model as shown below:
for obj in Category.objects.all():
print(obj.product_set.all())
Then, these below are outputted on console:
<QuerySet [<Product: Apple 10.00>, <Product: Orange 20.00>]>
<QuerySet [<Product: Carrot 30.00>]>
<QuerySet [<Product: Chicken 40.00>, <Product: Beef 50.00>]>
<QuerySet [<Product: Salmon 60.00>]>
<QuerySet [<Product: Apple 10.00>, <Product: Chicken 40.00>, <Product: Salmon 60.00>]>
Then, 6 SELECT
queries are run as shown below:
Next, I iterate Product
model from Category
model with select_related("product_set")
as shown below:
for obj in Category.objects.select_related("product_set").all():
print(obj.product_set.all())
Then, the error below occurs because select_related("categories")
cannot be used with Reverse Foreign Key:
django.core.exceptions.FieldError: Invalid field name(s) given in
select_related: 'product_set'. Choices are: (none)
Actually, there is no error if I use select_related()
with no argument as shown below:
# ↓ No argument
for obj in Category.objects.select_related().all():
print(obj.product_set.all())
Then, these below are outputted on console:
<QuerySet [<Product: Apple 10.00>, <Product: Orange 20.00>]>
<QuerySet [<Product: Carrot 30.00>]>
<QuerySet [<Product: Chicken 40.00>, <Product: Beef 50.00>]>
<QuerySet [<Product: Salmon 60.00>]>
<QuerySet [<Product: Apple 10.00>, <Product: Chicken 40.00>, <Product: Salmon 60.00>]>
But, 6 SELECT
queries are still run instead of 1 SELECT
query as shown below:
Next, I iterate Product
model from Category
model with prefetch_related("product_set")
as shown below:
for obj in Category.objects.prefetch_related("product_set").all():
print(obj.product_set.all())
Then, these below are outputted on console:
<QuerySet [<Product: Apple 10.00>, <Product: Orange 20.00>]>
<QuerySet [<Product: Carrot 30.00>]>
<QuerySet [<Product: Chicken 40.00>, <Product: Beef 50.00>]>
<QuerySet [<Product: Salmon 60.00>]>
<QuerySet [<Product: Apple 10.00>, <Product: Chicken 40.00>, <Product: Salmon 60.00>]>
Then, 2 SELECT
queries are run as shown below:
<Additional experiments>
For example, there are Country
, State
and City
models which have one-to-many relationship as shown below:
# "app/models.py"
from django.db import models
class Country(models.Model):
name = models.CharField(max_length=20)
def __str__(self):
return self.name
class State(models.Model):
country = models.ForeignKey(Country, on_delete=models.CASCADE)
name = models.CharField(max_length=20)
def __str__(self):
return self.name
class City(models.Model):
state = models.ForeignKey(State, on_delete=models.CASCADE)
name = models.CharField(max_length=20)
def __str__(self):
return self.name
And, there are Country
, State
and City
admins as shown below:
# "app/admin.py
from django.contrib import admin
from .models import Country, State, City
@admin.register(Country)
class CountryAdmin(admin.ModelAdmin):
list_display = ('id', 'name')
ordering = ('id',)
@admin.register(State)
class StateAdmin(admin.ModelAdmin):
list_display = ('id', 'name', 'get_country')
ordering = ('id',)
@admin.display(description='country')
def get_country(self, obj):
return obj.country
@admin.register(City)
class CityAdmin(admin.ModelAdmin):
list_display = ('id', 'name', 'get_state', 'get_country')
ordering = ('id',)
@admin.display(description='state')
def get_state(self, obj):
print(obj)
return obj.state
@admin.display(description='country')
def get_country(self, obj):
return obj.state.country
Then, Country
admin has 2 objects as shown below:
And, State
admin has 3 objects as shown below:
Then, City
admin has 6 objects as shown below:
<Forward Foreign Key>
Then, I iterate Country
model from City
model with select_related("state__country")
as shown below:
for obj in City.objects.all().select_related("state__country"):
print(obj.state.country)
Then, these below are outputted on console:
USA
USA
USA
USA
Japan
Japan
Then, 1 SELECT
query is run as shown below:
Next, I iterate Country
model from City
model with prefetch_related("state__country")
as shown below:
for obj in City.objects.all().prefetch_related("state__country"):
print(obj.state.country)
Then, these below are outputted on console:
USA
USA
USA
USA
Japan
Japan
Then, 3 SELECT
queries are run as shown below:
<Reverse Foreign Key>
Next, I iterate City
model from Country
model with prefetch_related("state_set__city_set")
as shown below:
for country_obj in Country.objects.all().prefetch_related("state_set__city_set"):
for state_obj in country_obj.state_set.all():
for city_obj in state_obj.city_set.all():
print(city_obj)
Then, these below are outputted on console:
San Francisco
Los Angeles
San Diego
Kansas City
Ginza
Akihabara
Then, 3 SELECT
queries are run as shown below:
Next, I iterate City
model from Country
model with prefetch_related("state_set__city_set")
with filter() instead of all() as shown below. *prefetch_related()
with filter()
doesn't work:
# Here
for country_obj in Country.objects.filter().prefetch_related("state_set__city_set"):
for state_obj in country_obj.state_set.filter(): # Here
for city_obj in state_obj.city_set.filter(): # Here
print(city_obj)
Then, these below are outputted on console:
San Francisco
Los Angeles
San Diego
Kansas City
Ginza
Akihabara
Then, 8 SELECT
queries are run as shown below:
So to reduce 8 SELECT
queries, I need to use Prefetch() with filter()
but I only know how to iterate State
model from Country
model with Prefetch()
and filter()
as shown below. *I asked the question on Stack Overflow about how to iterate City
model from Country
model with Prefetch()
and filter()
:
for country_obj in Country.objects.filter().prefetch_related(
Prefetch('state_set', # Here
queryset=State.objects.filter(),
to_attr='state_obj'
)
):
print(country_obj.state_obj)
Then, these below are outputted on console:
[<State: California>, <State: Missouri>]
[<State: Tokyo>]
Then, 2 SELECT
queries are run as shown below: