In ElasticSearch 7+ how do you search all text fields?

Question

I want to search for a word in documents I have stored in Elasticsearch 7.3

An example of what I would like on previous versions of Elasticsearch that worked is:

{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "_all": "oliver"
          }
        }
      ],
      "must_not": [],
      "should": []
    }
  },
  "from": 0,
  "size": 10,
  "sort": [],
  "aggs": {}
}

But this query will not work on Elasticsearch 7+ because _all has been removed. How can I write the query to accomplish the same thing now that the _all field is gone?

Note: I've read the suggestions to use copy_to to copy all fields into a custom field but that requires explicitly writing out each field to include in the all_fields field. Since I have lots of little fields I am trying to avoid this and instead replicate the behavior that being able to query _all allowed.

Quite

Answer 1

Its mentioned in ES documentation.

_all may no longer be enabled for indices created in 6.0+, use a custom field and the mapping copy_to parameter

So, what you need to do is create your own custom _all field, which you can name like all_fields etc and then use copy_to to copy all fields value in your own all_fields, so now as this custom field has all the fields value, so you can use it instead of _all fields.

Underwear

Using copy_to requires explicitly writing out each field to include in the all_fields field. Since I have lots of little fields I am trying to avoid this and instead replicate the behavior that being able to query _all allowed. – Quite

@John, I am afraid its no longer available as mentioned in the ES docs and you will have to use the workaround provided by ES which I already mentioned in my answer – Underwear

Done. Thanks for your help @opster-es-ninja – Quite

@Quite for your prompt action, this motivates more to help the community :) – Underwear

Answer 2

I have run into a similar situation. As far as I can tell, there are two ways you can search all text fields in a world without the _all field.

The most optimal way is probably to use a simple_query_string or query_string query. By default these will search all string (text or keyword) fields.

If, for some reason, you needed an explicit field that contained all text tokens like the _all field, you can create your own. You can use "copy_to": "_all" on explicitly mapped text or keyword fields to include known fields, but this only solves half the problem. To include unknown fields in this field, you would additionally need to create a dynamic template with the "copy_to": "_all" mapping. Dynamic templates specifically match dynamic fields, so if you combine this with the copy_to's on the explicitly mapped fields, you should have a field that contains all text tokens from both known and unknown fields. The reason this is less optimal is because this can have a significant impact on the size of your database. Here is a simple mapping example that should accomplish what you want:

  {
    "mappings": {
      "dynamic_templates": [
        {
          "all_text": {
            "match_mapping_type": "string",
            "mapping": {
              "copy_to": "_all",
              "type": "text"
            }
          }
        }
      ],
      "properties": {
        "_all": {
          "type": "text"
        },
        "email": {
          "type": "text",
          "copy_to": [
            "_all"
          ]
        },
        "name": {
          "type": "keyword",
          "copy_to": [
            "_all"
          ]
        }
      }
    }
  }

Pyrites