Logstash - remove deep field from json file

Question

I have JSON file that I'm sending to ES through logstash. I would like to remove 1 field ( It's deep field ) in the JSON - ONLY if the value is NULL.

Part of the JSON is:

"input": {
        "startDate": "2015-05-27",
        "numberOfGuests": 1,
        "fileName": "null",
        "existingSessionId": "XXXXXXXXXXXXX",
        **"radius": "null",**
        "nextItemReference": "51",
        "longitude": -99.12,
        "endDate": "2015-05-29",
        "thumbnailHeight": 200,
        "thumbnailWidth": 300,
        "latitude": 19.42,
        "numOfRooms": "1"
    },

Part in the logstash.conf file is :

if [input.radius] == "null" {
                mutate {
                        remove_field => [ "input.radius" ]
                }
        }

This is inside the filter of course.

How can I remove this field if the value is null?

September

Answer 1

Nested fields aren't referred with [name.subfield] but [field][subfield]. This should work for you:

if [input][radius] == "null" {
  mutate {
    remove_field => [ "[input][radius]" ]
  }
}

Note that if there is no "input" field, the [input][radius] reference will create an empty "input" dictionary. To avoid that you can do this:

if [input] and [input][radius] == "null" {
  mutate {
    remove_field => [ "[input][radius]" ]
  }
}

See the Logstash documentation for details and more examples.

Flak