Elasticsearch 2.0: how to delete by query in Java

Question

I am trying to upgrade to ES 2.0. I have downloaed ES 2.0 and installed it on my Windows machine.

In my pom.xml, I have the following:

<dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch</artifactId>
    <version>2.0.0-rc1</version>
</dependency>

<dependency>
    <groupId>org.elasticsearch.plugin</groupId>
    <artifactId>delete-by-query</artifactId>
    <version>2.0.0-rc1</version>
</dependency>

In my Java code, I did delete by query in the following way when using ES 1.7.3:

    StringBuilder b = new StringBuilder("");
    b.append("{");
    b.append("  \"query\": {");  
    b.append("      \"term\": {");
    b.append("          \"category\": " + category_value );
    b.append("      }");
    b.append("  }");
    b.append("}");

    client = getClient(); 

    DeleteByQueryResponse response = client.prepareDeleteByQuery("myindex")
                .setTypes("mydocytype")
                .setSource(b.toString())
                .execute()
                .actionGet();

I am hoping to replace this:

    DeleteByQueryResponse response = client.prepareDeleteByQuery("myindex")
                .setTypes("mydocytype")
                .setSource(b.toString())
                .execute()
                .actionGet();

with ES 2.0 way. Googled but failed to find an example for it. The online API documentation seems too abstract to me. How can I do it?

Another question: Do I have to install delete-by-query plugin in Elasticsearch server?

Thanks for any pointer!

UPDATE

I followed Max's suggestion, and here is what I have now:

First, when create the client, make settings look like the following:

Settings settings = Settings.settingsBuilder()
                        .put("cluster.name", "mycluster")
                        .put("plugin.types", DeleteByQueryPlugin.class.getName())
                        .build();

Second, at the place doing delete-by-query:

    DeleteByQueryResponse rsp = new DeleteByQueryRequestBuilder(client, DeleteByQueryAction.INSTANCE)
    .setIndices("myindex")
    .setTypes("mydoctype")
    .setSource(b.toString())
    .execute()
    .actionGet();

I also installed delete by query plugin by running the following in the root directory of ES:

bin\plugin install delete-by-query

I get errors if I do not install this plugin.

After all these steps, ES related parts work just fine.

Webb

Answer 1

I believe you can use this:

     DeleteByQueryResponse rsp = new DeleteByQueryRequestBuilder(client, DeleteByQueryAction.INSTANCE)
            .setTypes("mydocytype")
            .setSource(b.toString())
            .execute()
            .actionGet();

You have to add plugin type to your settings:

     Settings settings = Settings.settingsBuilder()
                         .put("plugin.types", DeleteByQueryPlugin.class.getName())

If you have remote server you have to install the plugin.

Hailstorm

Hello Max, thanks for helping me out! What do you mean by "using local node"? I have everything on my notebook. The Java web app runs on port 8080 and ES runs on port 9300. I have to manually start the two one by one. Another thing: Your settings statement is in error. I updated my code, and always got null pointer error. – Webb

Hello, with the local node I meant using embedded ES node (elastic.co/guide/en/elasticsearch/client/java-api/current/…). When you use this you don't have to start another elasticsearch client. I think you are connecting to your elasticsearch node using transport client(elastic.co/guide/en/elasticsearch/client/java-api/current/…). Where exactly do you get null pointer ? – Hailstorm

Ok I see that even if you don't use embedded node you have to put plugin.types. I am editing my comment for this. – Hailstorm

Max, thanks for follow-up and making your answer clearer. Really really really appreciate it! I got my code working now due to your help. Best. – Webb

I get an NPE when I use the delete-by-query plugin with the TransportClient. Happens regardless if I add the plugin type to the settings or not. – Inhibit

Answer 2

plugin.types have been deprecated in ES 2.1.0 (source). So the accepted solution will result in a NullPointerException.

The solution is to use the addPlugin method:

Client client = TransportClient.builder().settings(settings())
                .addPlugin(DeleteByQueryPlugin.class)
                .build()
                .addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("host",9300));

Gefell

What is the purpose of "adding the plugin" on the client side as well? – Editorial

Answer 3

From Elastic 5 in onwards...

final BulkIndexByScrollResponse response = DeleteByQueryAction.INSTANCE.newRequestBuilder(super.transportClient)
                    .filter(
                            QueryBuilders.boolQuery()
                                    .must(QueryBuilders.termQuery("_type", "MY_TYPE")) // Trick to define and ensure the type.
                                    .must(QueryBuilders.termQuery("...", "...")))
                    .source("MY_INDEX")
                    .get();

    return response.getDeleted() > 0;

Oficial documentation

Dismiss

Dani, thanks for the info. Good to know for upgrade. Cheers. – Webb

Answer 4

firstly: add elasticsearch-2.3.3/plugins/delete-by-query/delete-by-query-2.3.3.jar to build path.

then:

Client client = TransportClient.builder().settings(settings)
                .addPlugin(DeleteByQueryPlugin.class)
                .build()
                .addTransportAddress(new InetSocketTransportAddress(
                        InetAddress.getByName("192.168.0.224"), 9300));

Bengali