elasticsearch-painless - Manipulate date

Question

I am trying to manipulate date in elasticsearch's scripting language painless. Specifically, I am trying to add 4 hours, which is 14,400 seconds.

{
  "script_fields": {
    "new_date_field": {
      "script": {
        "inline": "doc['date_field'] + 14400"
      }
    }
  }
}

This throws Cannot apply [+] operation to types [org.elasticsearch.index.fielddata.ScriptDocValues.Longs] and [java.lang.Integer].

Thanks

Deliberation

Answer 1

The solution was to use .value

{
  "script_fields": {
    "new_date_field": {
      "script": {
        "inline": "doc['date_field'].value + 14400"
      }
    }
  }
}

However, I actually wanted to use it for reindexing, where the format is a bit different. Here is my version for manipulating time in the _reindex api

POST _reindex
{
  "source": {
    "index": "some_index_v1"
  },
  "dest": {
    "index": "some_index_v2"
  },
  "script": {
    "inline": "def sf = new SimpleDateFormat(\"yyyy-MM-dd'T'HH:mm:ss\"); def dt = sf.parse(ctx._source.date_field); def calendar = sf.getCalendar(); calendar.setTime(dt); def instant = calendar.toInstant(); def localDateTime = LocalDateTime.ofInstant(instant, ZoneOffset.UTC); ctx._source.date_field = localDateTime.plusHours(4);"
  }
}

Here is the inline script in a readable version

def sf = new SimpleDateFormat(\"yyyy-MM-dd'T'HH:mm:ss\");
def dt = sf.parse(ctx._source.date_field);
def calendar = sf.getCalendar();
calendar.setTime(dt);
def instant = calendar.toInstant();
def localDateTime = LocalDateTime.ofInstant(instant, ZoneOffset.UTC);
ctx._source.date_field = localDateTime.plusHours(4);

Here is the list of functions supported by painless, it was painful.

Deliberation

This was a very helpful lead for me to deal with a date string format with fractional seconds (2018-05-28T13:49:40.03Z). The link to the repo helped me learn that java.time classes (notably Instant) was supported in a painless script so I could write def existLastSeen = Instant.parse(ctx._source.last_seen_ts); def thisLastSeen = Instant.parse(params.last_seen_ts); if (thisLastSeen.isAfter(existLastSeen)) { ctx._source.last_seen_ts = params.last_seen_ts; } – Telpherage

Thanks for the link to the painless supported functions, very useful! – Nieberg

For some reason, the single quote around T character in the date format gets stripped when read by ES. I am using def df = DateTimeFormatter.ofPattern(\"yyyy-MM-dd'T'HH:mm:ss\"); – Nepean

It was PAINFUL! – Gallardo

Answer 2

An addition. Converting date to a string, your first part I believe, can be done with:

def dt = String.valueOf(ctx._source.date_field);

Just spent a couple of hours playing with this.. so I can concantenate a date field (in UTC format with 00:00:00 added).. to a string with the time, to get a valid datetime to add to ES. Don't ask why it was split.. its an old Oracle system

Tellurion