Elasticsearch returns total 624 but hits array is empty

Question

I have this query with a strange result: it returns a total of 612 documents but no hits.

This is the query:

{
  "from": 900,
  "size": 30,
  "query": {
    "filtered": {
      "filter": {
        "bool": {
          "must": [
            {
              "term": {
                "user_id": "145698489"
              }
            }
          ],
          "should": [
            {
              "bool": {
                "must": [
                  {
                    "range": {
                      "date_created": {
                        "to": "2016-07-30T20:11:09.176-03:00",
                        "gte": "2016-07-07T23:39:45.530-03:00"
                      }
                    }
                  }
                ],
                "must_not": [
                  {
                    "term": {
                      "resource.object.label": "hidden"
                    }
                  }
                ],
                "should": []
              }
            }
          ]
        }
      }
    }
  }
}

And this is the result:

{
    "_shards": {
        "failed": 0,
        "successful": 1,
        "total": 1
    },
    "hits": {
        "hits": [],
        "max_score": 1,
        "total": 612
    },
    "timed_out": false,
    "took": 73
}

But if I change the date ranges I get the results correctly.

Why ES is doing this?

Loathsome

Answer 1

Because you set from to 900. If you have less than 900 hits it won't return hits.

Kingkingbird

Answer 2

in my case this happened because i set the limit to zero, lol.

Impulsive

Welcome! This is not really an answer to the question. Please read this on how to write good answers. – Diamagnetism

this is a fine answer to the question. It could easily help someone who does the same in the future – Vacuva

Answer 3

You should try to set from :0.in your query. Also important to know, is that there were some changes to ES recently about how it handles scrolling. Hence the spot where you should put scrol is important, try to set one next_scroll = results['_scroll_id'] out side while loop and the other scroll next_scroll = results['_scroll_id'] at the end of your while loop code line. After i have changed the scrol spot it works for me.

Nonentity

Answer 4

We also hit this, and to resolve it we needed to remove the excludes cluase from the _source

_source: {
  excludes: %w{embedding score tags}
},

Vacuva