Delete data older than 10 days in elasticsearch

Question

I am new to elasticsearch and I want to delete documents in my elasticsearch index which are older than 10 days. I want to keep only last 10 days of data.So is there any way to delete last 11nth day index automatically. What I have tried..

DELETE logstash-*/_query
{
 "query": {
   "range": {
     "@timestamp": {
       "lte": "now-10d"
      }
    }
  }
}

Error I'm getting while running on kibana dev tools

{
"error": "Incorrect HTTP method for uri [/logstash-*/_query?pretty] and method [DELETE], 
allowed: [POST]",
"status": 405
}

Please help to resolve this issues.

Flaunt

Answer 1

You need to leverage the Delete by Query Endpoint, like this:

use POST           use this endpoint
 |                      |
 V                      V
POST logstash-*/_delete_by_query
{
 "query": {
   "range": {
     "@timestamp": {
       "lte": "now-10d"
      }
    }
  }
}
  ^
  |
the query part is fine !!

Whizbang

Thanks for your response but its not working.... { "took" : 1, "timed_out" : false, "total" : 0, "deleted" : 0, "batches" : 0, "version_conflicts" : 0, "noops" : 0, "retries" : { "bulk" : 0, "search" : 0 }, "throttled_millis" : 0, "requests_per_second" : -1.0, "throttled_until_millis" : 0, "failures" : [ ] } . I'm getting such output and its not reflecting any changes. – Flaunt

Well with the response you showed that @Whizbang 's answer fixed the original issue. Can you verify that there are even documents in that time range? – Meteor

That means no documents matched the range. Replace _delete_by_query with _search and no documents should come up. – Whizbang

Answer 2

I am describe diffent approach, then @Val have suggested. You can create 10 indexes (index per day) and each day delete one of the indexes - oldest one.

• Pros: it is very easy to delete or archive old data
• Cons: you need to rewrite your queries if you need to search all days data.

Animalism

You can do same thing using ILM policy instead of through API each time. – Designedly