Is there is any way to iterate elastic array document like other programming language with script

Question

Mapping

{
"supply": {
  "properties": {
    "rotation_list": {
      "type": "nested",
      "properties": {
        "project_end_date": {
          "type": "nested",
          "properties": {
            "end_date": {
              "type": "date",
              "format": "yyyy-MM-ddTHH:mm:ss"
            }
          }
        },
        "total_days": {
          "type": "integer"
        }
      }
    }
  }
}}

Data

{"rotation_list": [
{
  "project_end_date": [
    {
      "end_date": "2020-08-07"
    },
    {
      "end_date": "2020-06-07"
    }
  ],
  "total_days": 23
},
{
  "project_end_date": [
    {
      "end_date": "2020-08-07"
    }
  ],
  "total_days": 26
}]}

query

{"query": {
"bool": {
  "filter": {
    "bool": {
      "must": [
        {
          "nested": {
            "path": "rotation_list.project_end_date",
            "query": {
              "script": {
                "script": {
                  "lang": "groovy",
                  "inline": "import  org.elasticsearch.common.logging.*;logger=ESLoggerFactory.getLogger('myscript');def ratable =false;logger.info(doc['rotation_list.project_end_date.end_date'].values)"
                }
              }
            }
          }
        }
      ]
    }
  }
}}}

Log result

[INFO ][myscript] [1596758400000] [INFO ][myscript] [1591488000000] [INFO ][myscript] [1596758400000]

I am not sure why this is happning. Is there is any way to iterate like [1596758400000, 1591488000000] and [1596758400000]. Data is saved like this as well. I have mentioned in the mapping as well nested type. Not sure why this is returning like this. Is there is any way to iterate like original document i have indexed.

Faux

Elastic raw result is always in JSON format. It can be accessed/iterated the way any other JSON is. – Inexpiable

With the script when i am iterating then why it's giving value as [1596758400000],[1591488000000], [1596758400000]. Why not like [1596758400000, 1591488000000] and [1596758400000]. Is there is any way to iterate like this. @OpsterESNinjaNishant – Faux

Answer 1

It's impossible to access a nested doc's nested neighbor in a script query due to the nature of nested whereby each (sub)document is treated as a separate document -- be it on the top level or within an array of objects like your rotation_list.project_end_date.

The only permissible situation of having access to the whole context of a nested field is within script_fields -- but you unfortunately cannot query by them -- only construct them on the fly & retrieve them:

Using your mapping from above

GET supply_nested/_search
{
  "script_fields": {
    "combined_end_dates": {
      "script": {
        "lang": "painless",
        "source": "params['_source']['rotation_list'][0]['project_end_date']"
      }
    }
  }
}

---

Iterating within a script query be possible only if rotation_list alone were nested but not project_end_date. Using 7.x here:

PUT supply_non_nested
{
  "mappings": {
    "properties": {
      "rotation_list": {
        "type": "nested",
        "properties": {
          "project_end_date": {
            "type": "object",
            "properties": {
              "end_date": {
                "type": "date",
                "format": "yyyy-MM-dd"
              }
            }
          },
          "total_days": {
            "type": "integer"
          }
        }
      }
    }
  }
}

Sync a doc:

POST supply_non_nested/_doc
{
  "rotation_list": [
    {
      "project_end_date": [
        {
          "end_date": "2020-08-07"
        },
        {
          "end_date": "2020-06-07"
        }
      ],
      "total_days": 23
    },
    {
      "project_end_date": [
        {
          "end_date": "2020-08-07"
        }
      ],
      "total_days": 26
    }
  ]
}

Query using painless instead of groovy because it's more secure & less verbose in this case:

GET supply_non_nested/_search
{
  "query": {
    "bool": {
      "filter": {
        "bool": {
          "must": [
            {
              "nested": {
                "path": "rotation_list",
                "query": {
                  "script": {
                    "script": {
                      "lang": "painless",
                      "inline": "Debug.explain(doc['rotation_list.project_end_date.end_date'])"
                    }
                  }
                }
              }
            }
          ]
        }
      }
    }
  }
}

yielding

...
"reason": {
          ...
          "to_string": "[2020-06-07T00:00:00.000Z, 2020-08-07T00:00:00.000Z]",
          "java_class": "org.elasticsearch.index.fielddata.ScriptDocValues$Dates",
        }
...

---

It's not exactly clear from your snippet what you were trying to achieve in the query. Can you elaborate?

Muumuu

Hey! Joe, When I use script_fields I am able to iterate over nested objects, access nested fields, construct a custom field, and retrieve them, but unfortunately unable to sort it based on that custom field. I tried using script_score and sort based on a custom field but was not able to access nested objects. Your help will be much appreciated. – Beiderbecke

Sorry to miss I am using 6.8.0 version – Beiderbecke

Hey - see if this answer of mine helps? https://mcmap.net/q/1781667/-how-to-sort-the-result-set-based-on-the-minimum-value-of-a-nested-field-in-elasticsearch – Muumuu

Hey @Joe, its a bit different than what you have answered. I have just posted a question, could you please check it. #70833983 – Beiderbecke

Gotcha. I've your new question over there. – Muumuu