How can I filter a Haystack SearchQuerySet for None on an IntegerField

Question

This is driving me a bit mad but seems like it should be simple.

I'm using Django and Haystack and have a search index including an IntegerField which allows null. This is based on a related model in Django, but I don't think this matters. eg:

class ThingIndex(indexes.ModelSearchIndex, indexes.Indexable):
    group = indexes.IntegerField(model_attr='group__id', null=True)

    class Meta:
        model = Thing

I sometimes want my Haystack query to return items with None/Null for this field, so I'm filtering in the search form's __init__, but I can't get a query to do this. The most obvious way I tried was:

self.searchqueryset.filter(group__isnull=True)  # how to do it on a regular Django queryset

But this returns no records.

Right now I'm working around it with this:

self.searchqueryset.exclude(group__in=range(1,100))

Which works, but obviously isn't the way it should be done :)

Can anyone help?

Thanks!

Tamper

Answer 1

I feel this question was not answered. It seems the op was asking how to filter for null entries using haystack.query.SearchQuerySet with an ElasticSearch backend.

In the example above, replace

self.searchqueryset.filter(group__isnull=True)

with

self.searchqueryset.filter(_missing_='group')

Not intuitive, but its the only way I have gotten this to work so far.

Pave

Thanks for this. If possible, please add the source for this answer. – Prittleprattle

@Pave thanks, this works for me with .exclude too. Do you know how can I can search for objects excluding those that have field a missing and field b missing? Example with obviously invalid syntax: orphans = children.exclude(_missing_='mother' AND _missing_='father') - thanks! – Apsis

Answering myself: solved using SQ: django-haystack.readthedocs.io/en/v2.4.1/… – Apsis

Answer 2

If you are using ElasticSearch, the solution can be done without patching, just using native ElasticSearch:

from haystack.inputs import Raw
self.searchqueryset.exclude(group = Raw("[* TO *]"))

Other way around, filter all documents that have non-emtpy group field:

from haystack.inputs import Raw
self.searchqueryset.filter(group = Raw("[* TO *]"))

This could work for SOLR too and for other Haystack backends applying the same concept but with specific syntax of the backend search language.

References:

• Elastic search query syntax
• Haystack backend implementation for elasticsearch
• Haystack Raw input field
• Thanks to @AlexParakhnevich who provided this link which served me as an inspiration.
• Good to know that Elasticsearch also supports special and more robust "missing" filter

Bisectrix

nice.. just one thing you have a TYPO, it should be self.searchqueryset.exclude(group = Raw("[* TO *]")) – Participial

Answer 3

If you're using SOLR, you'd probably like to have a look at the following links:

1) https://github.com/toastdriven/django-haystack/commit/9332a91a7f0e4b33d7e20aa892d156305c12dfe3
2) https://github.com/toastdriven/django-haystack/issues/163

There's a patch for SOLR, allowing such queries, but for other backends there's probably none.

Constitutionalism

Thanks! Really useful. Not using SOLR, we're using ElasticSearch. Oh well. – Tamper