Elasticsearch autocomplete search on array field

Question

I am working on autocomplete suggestion on document field that has array of type string. My document is like below;

{

    "title": "Product1",
    "sales": "6",
    "rating": "0.0",
    "cost": "45.00",
    "tags": [
        "blog",
        "magazine",
        "responsive",
        "two columns",
        "wordpress"
    ],
    "category": "wordpress",
    "description": "Product1 Description",
    "createDate": "2013-12-19"
}

{

    "title": "Product1",
    "sales": "6",
    "rating": "0.0",
    "cost": "45.00",
    "tags": [
        "blog",
        "paypal",
        "responsive",
        "skrill",
        "wordland"
    ],
    "category": "wordpress",
    "description": "Product1 Description",
    "createDate": "2013-12-19"
}

I am performing autocomplete search on tags field. My query is like;

query: {
                    query_string: {
                        query: "word*",
                        fields: ["tags"]
                    }
                },
                facets: {
                    tags: {
                        terms: {
                            field: "tags"
                        }
                    }
                }

When user type "word" I want to display "wordland" and "wordpress". However, I couldn't manage to do that.

Could you please help on this?

Thanks

Filigree

Answer 1

Have you tried completion suggest? One way to solve your problem is as follows:

1) Create the index:

curl -XPUT "http://localhost:9200/test_index/"

2) Create the mapping, using the completion suggester type:

curl -XPUT "http://localhost:9200/test_index/product/_mapping" -d'
{
   "product": {
      "properties": {
         "category": {
            "type": "string"
         },
         "cost": {
            "type": "string"
         },
         "createDate": {
            "type": "date",
            "format": "dateOptionalTime"
         },
         "description": {
            "type": "string"
         },
         "rating": {
            "type": "string"
         },
         "sales": {
            "type": "string"
         },
         "tags": {
            "type": "string"
         },
         "title": {
            "type": "string"
         },
         "suggest": {
            "type": "completion",
            "index_analyzer": "simple",
            "search_analyzer": "simple",
            "payloads": false
         }
      }
   }
}'

3) Add your documents:

curl -XPUT "http://localhost:9200/test_index/product/1" -d'
{
   "title": "Product1",
   "sales": "6",
   "rating": "0.0",
   "cost": "45.00",
   "tags": [
      "blog",
      "magazine",
      "responsive",
      "two columns",
      "wordpress"
   ],
   "suggest": {
      "input": [
         "blog",
         "magazine",
         "responsive",
         "two columns",
         "wordpress"
      ]
   },
   "category": "wordpress",
   "description": "Product1 Description",
   "createDate": "2013-12-19"
}'

curl -XPUT "http://localhost:9200/test_index/product/2" -d'
{

    "title": "Product2",
    "sales": "6",
    "rating": "0.0",
    "cost": "45.00",
    "tags": [
        "blog",
        "paypal",
        "responsive",
        "skrill",
        "wordland"
    ],
   "suggest": {
      "input": [
         "blog",
        "paypal",
        "responsive",
        "skrill",
        "wordland"
      ]
   },
    "category": "wordpress",
    "description": "Product1 Description",
    "createDate": "2013-12-19"
}'

4) And then query using the _suggest endpoint:

curl -XPOST "http://localhost:9200/test_index/_suggest" -d'
{
    "product_suggest":{
        "text":"word",
        "completion": {
            "field" : "suggest"
        }
    }
}'

and you will get the results back that you expected:

{
   "_shards": {
      "total": 2,
      "successful": 2,
      "failed": 0
   },
   "product_suggest": [
      {
         "text": "word",
         "offset": 0,
         "length": 4,
         "options": [
            {
               "text": "wordland",
               "score": 1
            },
            {
               "text": "wordpress",
               "score": 1
            }
         ]
      }
   ]
}

This solution could be refined a bit, of course, particularly by pruning some duplicate data, but this should point you in the right direction.

Receptionist

Is there any way to achieve this on a pre-existing index, with no need for reseeding. This approach that you have described, leads to duplication of data, which i don't want. – Kairouan

Yes, you can use ngrams. I like that method better anyway. If you'll post another question about it I'll give you an example. – Receptionist

This solution is not working for me in ES 5.3. The playload option is not supported anymore, and the output contains the full document and not the tag that matched the term – Coraciiform

Well, I was tricked by the new behavior of completion suggest in ES 5. ES5 does not deduplicate results and the completion suggester cannot be used in this fashion anymore. There is an issue: github.com/elastic/elasticsearch/issues/22912 and someone posted a plugin to have the same behavior in ES <5 and in ES 5(see in the issue) – Coraciiform

@SloanAhrens the completion suggester from ES is not enough for me, I need more control on what matches or not, and I have opened a very similar question https://mcmap.net/q/669593/-elasticsearch-search-suggestion-on-array-field-with-partial-edge-ngram-completion/2832282, if you have some ideas that would be great! – Scow