exclude _id and _index field in elasticsearch result data

Question

there are 5 fields in each document if simply hit the api. but i want only these two fields(user_id and loc_code) so I mentioned in fields list. but still it return some unnecessary data like _shards,hits,time_out etc.

making POST request in postman plugin in chrome using below query

<:9200>/myindex/mytype/_search
{
    "fields" : ["user_id", "loc_code"],
    "query":{"term":{"group_id":"1sd323s"}}
}   

// output

 {
        "took": 17,
        "timed_out": false,
        "_shards": {
            "total": 5,
            "successful": 5,
            "failed": 0
        },
        "hits": {
            "total": 323,
            "max_score": 8.402096,
            "hits": [
                {
                    "_index": "myindex",
                    "_type": "mytype",
                    "_id": "<someid>",
                    "_score": 8.402096,
                    "fields": {
                        "user_id": [
                            "<someuserid>"
                        ],
                        "loc_code": [
                            768
                        ]
                    }
                },
               ...
            ]
        }
    }

but I want only documents fields(two mentioned fields) neither I want _id,_index,_type. is there any way to do so

Prosaism

Answer 1

A solution that may not be complete but helps a lot is to use filter_path. For example, suppose we have the following content in an index:

PUT foods/_doc/_bulk
{ "index" : { "_id" : "1" } }
{ "name" : "chocolate cake", "calories": "too much" }
{ "index" : { "_id" : "2" } }
{ "name" : "lemon pie", "calories": "a lot!"  }
{ "index" : { "_id" : "3" } }
{ "name" : "pizza", "calories": "oh boy..."  }

A search like this...

GET foods/_search
{
  "query": {
    "match_all": {}
  }
}

...will yield a lot of metadata:

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "foods",
        "_type" : "_doc",
        "_id" : "2",
        "_score" : 1.0,
        "_source" : {
          "name" : "lemon pie",
          "calories" : "a lot!"
        }
      },
      {
        "_index" : "foods",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "name" : "chocolate cake",
          "calories" : "too much"
        }
      },
      {
        "_index" : "foods",
        "_type" : "_doc",
        "_id" : "3",
        "_score" : 1.0,
        "_source" : {
          "name" : "pizza",
          "calories" : "oh boy..."
        }
      }
    ]
  }
}

But if we give the search URL the parameter filter_path=hits.hits._source...

GET foods/_search?filter_path=hits.hits._source
{
  "query": {
    "match_all": {}
  }
}

...it will only return the source (although still deeply nested):

{
  "hits" : {
    "hits" : [
      {
        "_source" : {
          "name" : "lemon pie",
          "calories" : "a lot!"
        }
      },
      {
        "_source" : {
          "name" : "chocolate cake",
          "calories" : "too much"
        }
      },
      {
        "_source" : {
          "name" : "pizza",
          "calories" : "oh boy..."
        }
      }
    ]
  }
}

You can even filter for fields:

GET foods/_search?filter_path=hits.hits._source.name
{
  "query": {
    "match_all": {}
  }
}

...and you will get this:

{
  "hits" : {
    "hits" : [
      {
        "_source" : {
          "name" : "lemon pie"
        }
      },
      {
        "_source" : {
          "name" : "chocolate cake"
        }
      },
      {
        "_source" : {
          "name" : "pizza"
        }
      }
    ]
  }
}

And you can do a lot more if you will: just check the documentation.

Papert

Answer 2

You may be able to use the GET api instead. Try that with something like:

/myindex/mytype/<objectId>/_source

In your results you will get just the _source.

See: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-get.html

Well, this assumes you know the id of the document. I'm not sure if you can exclude the meta-data when using the search api.

Perhaps: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-source-filtering.html

Melmon

This doesn't really answer the question. I can't imagine why the ES team doesn't allow full control of the output. It really should be possible to exclude output fields like _index, _type, _id, _score and _sort – Elusion