Updating indexed document in Elasticsearch

Question

I am trying to understand how you update an indexed document in Elasticsearch. I don't understand how it works? What is the ctx that the API is referring to doing? Let say you have a document with nested documents what do you have to do to update it?

And what is the difference between deleting the document and then index the "updated" version, vs a plain update?

Rompers

Answer 1

The update request retrieve source from Elasticsearch, modifies it and indexes it back to Elasticsearch. If you already have a copy of the document using update makes little sense. It would be generally faster to just index the new version. However, if you don't have the document readily available but you know which changes you would like to make to the document, it might be more efficient to use update.

For example, if I don't have a copy of the car document, but I want to add a new creator I can do something like this:

curl -XDELETE localhost:9200/test

curl -XPUT localhost:9200/test -d '{
    "settings": {
        "index.number_of_shards": 1,
        "index.number_of_replicas": 0
    },
    "mappings": {
        "car": {
            "properties": {
                "creators" : {
                    "type": "nested",
                    "properties": {
                        "name": {"type":"string"}
                    }
                }
            }
        }
    }
}
'

curl -XPOST localhost:9200/test/car/1 -d '{
    "creators": [{
        "name": "Steve"
    }]
}
'

echo
curl -XPOST localhost:9200/test/car/1/_update -d '{
    "script" : "ctx._source.creators += new_creator",
    "params" : {
        "new_creator" : {"name": "John"}
    }
}'

echo
curl "localhost:9200/test/car/1?pretty=true"
echo

In the update script ctx is a special variable that allows you to access the source of the object that you want to update. The ctx._source is a writable version of the source. You can modify this document in the script and the modified source will be persisted as the new version of the document.

Blister

Thanks for another great response :) So you would make the update method in a Java application by first deleting the document and then index it again. – Rompers

No need to delete the document first. You can simply index it. If a document with the same id already exists it will be automatically deleted and replaced with the new version. – Blister

I got an error executing the update script... in ES v1.3.1. Any ideas? {"error":"ElasticsearchIllegalArgumentException[failed to execute script]; nested: ScriptException[dynamic scripting for [mvel] disabled]; ","status":400} – Charie

@Charie See elasticsearch.org/guide/en/elasticsearch/reference/current/… – Blister

@Blister Great answer! BTW is the source language of the script groovy or mvel? I think the default language was mvel at the time of posting, now the default is groovy (if it is not changed explicitly in ES configuration). – Sarina

Is it necessary to send curl -XPUT request for mappings again while Index is already exist? – Pang

@batmaci deletion and creation of the index is just to show complete snippet that works – Blister

Do you happen to know how if its possible to 'update' on of these nested documents (ie the car already has 5 creators, one of which is John, and i want to rename John to Johnny withouth knowing the other creators? – Reticular

@Reticular nested documents are extracted from source. You can update any part of source, which means you can update nested documents as well. – Blister

Answer 2

Here is how to update a document with nested customer documents in ElasticSearch 7.3:

POST /myindex/_doc/mypartid/_update
{
    "script" : {
        "source": "ctx._source.customers.add(params.newcust)",
        "params" : {
            "newcust" : {"customer": "cust3"}
        }
    }
}

Results in:

GET /myindex/_doc/mypartid
{
    "_index": "myindex2",
    "_type": "_doc",
    "_id": "mypartid",
    ...
    "_source": {
        "part": "my part",
        "customers": [
            {"customer": "cust1"},
            {"customer": "cust3"}
        ],
        "machines": [
            {"machine": "mach7"},
            {"machine": "mach2"}
        ]
    }
}

So, for ES7+ the URL has changed as have the way you do scripts and which operations an array can have (+= does not work).

Valeric

This only answers the question of how you add a new nested doc. But how would update specific fields in a given nested doc? How would ES even identify the nested doc to update in the update query? – Davao

With a script which loops over the array and finds the right entry. There may be a nicer way with some ArrayList methods. – Valeric

With a script which loops over the array and finds the right entry. There may be a nicer way with some ArrayList methods. – Valeric