Haystack and Elasticsearch: Limit number of results

Question

I have 2 servers with Haystack:

• Server1: This has elasticsearch installed
• Server2: This doesn't have elasticsearch, the queries are made to Server1

My issue is about pagination when I make queries from Server2 to Server1:

• Server2 makes query to Server1
• Server1 send all the results back to Server2
• Server2 makes the pagination

But this is not optimal, if the query return 10.000 objects, the query will be slow.

I know that you can send to elasticsearch some values in the query (size, from and to) but I don't know if this is possible using Haystack, I've checked documentation and googled it and found nothing.

• How could I configure the query in Haystack to receive the results 10 by 10 ?

Edit

• Is possible that if I make SearchQuerySet()[10000:10010] it will only ask for this 10 items ?
• Or it will ask for all the items and then filter them ?

Edit2

I found this on Haystack Docs:

• SearchQuery API - set_limits

it seems a function to do whatt I'm trying to do:

Restricts the query by altering either the start, end or both offsets.

And then I tried to do:

from haystack.query import SearchQuerySet

sqs = SearchQuerySet()
sqs.query.set_limits(low=0, high=4)
sqs.filter(content='anything')

The result is the full list, like I never add the set_limit line

• Why is not working ?

Parts

Answer 1

Haystack works kinda different from Django ORM. After limiting the queryset, you should call get_results() in order to get limited results. This is actually smart, because it avoids multiple requests from Elastic.

Example:

# Assume you have 800 records.
sqs = SearchQuerySet()
sqs.query.set_limits(low=0, high=4)
len(sqs)  # Will return 800 records
len(sqs.get_results())  # Will return first 4 records.

Hope that it helps.

Mastersinger

I have to try this, I fought with the code a lot back then, and didn't find info to achieve it. Thanks !! – Parts

I get this error - AttributeError: 'SearchQuerySet' object has no attribute 'get_results' – Needham

last line should be len(sqs.query.get_results()) – Needham

Answer 2

Adding on to the Yigit answer, if you want to have these offsets on filtered records just add filter condition when you form the SearchQuerySet.

Also remember once the limits are set you can't change them by setting them again. You would need to form the SearchQuerySet() again, or there is a method to clear the limits.

results = SearchQuerySet().filter(content="keyword")
#we have a filtered resultSet now let's find specific records
results.query.set_limits(0,4)
return results.query.get_results()

Needham