Elasticsearch HTTP authentication in Spring

Question

I want to access a remote elasticsearch which is protected by a username and password. https://[username]:[password]@aws-eu-west-1-portal1.dblayer.com:11109/

In Spring using the XML config I was able to access my localhost elastic as shown below

<!-- ElasticSearch -->
<elasticsearch:repositories base-package="be.smartsearch.service.repository.elasticsearch" />

<elasticsearch:transport-client id="esClient" cluster-nodes="localhost:9300" />

<bean id="elasticsearchTemplate" class="org.springframework.data.elasticsearch.core.ElasticsearchTemplate">
  <constructor-arg name="client" ref="esClient" />
</bean>

The only usefull documentation I found so far is for PHP: https://www.elastic.co/guide/en/elasticsearch/client/php-api/current/_security.html

How can I connect to a remote elasticsearh with credentials in Spring data with the XML config?

UPDATE 1

In Mongo I was able to do it by the following method

<!-- Mongo -->
<mongo:mongo host="${mongo.host}" port="${mongo.port}"/>

<mongo:db-factory dbname="SmartSearchAfterDemo" mongo-ref="mongo" username="${mongo.user}" password="${mongo.password}"/>
<!--<mongo:db-factory dbname="${mongo.dbname}" mongo-ref="mongo"/> -->

<bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate">
    <constructor-arg name="mongoDbFactory" ref="mongoDbFactory"/>
</bean>

<mongo:repositories base-package="be.smartsearch.service.repository.mongo"/>

Seagraves

What do you mean by "this is for PHP"? The post is entitled "HTTP authentication" and the code shown uses Transport client (binary protocol)? – Singband

@GQuintana The code they show is based on he Elasticsearch-PHP client and the Spring data client doesn't provide the functionality to perform it in teh same way as the PHP client – Seagraves

The difference is PHP Client uses HTTP protocol and Java Client uses Transport protocol. – Singband

@GQuintana Do you have any idea how to connect to https://[username]:[password]@aws-eu-west-1-portal1.dblayer.com:11109/ in java? – Seagraves

Answer 1

Spring Data Elasticsearch is base on the official Elasticsearch Java Client which uses the binary Transport procol (not the REST HTTP procol like PHP).

If you're using Shield to secure your Elasticsearch, then you can set the user/password on the Transport client/Transport procol

TransportClient client = TransportClient.builder()
    .addPlugin(ShieldPlugin.class)
    .settings(Settings.builder()
        .put("cluster.name", "yourcluster")
        .put("shield.user", "youruser:yourpassword")
        ...
        .build())

If you wan't to use the HTTP protocol from Java code then there are to community clients:

• Jest which supports HTTP authentication
• Elasticsearch HTTP which is pretty new

But these solutions are not compatible with Spring Data

Singband

Thanks for the clear answer, I have no idea if it's using Shield because this is a rented server from compose.io/elasticsearch and they just provided me the credentials – Seagraves

Answer 2

If you are using ElasticSearch's Docker image, it comes with X-Pack:

• Elastic.co: Install ElasticSearch with Docker
• Elastic.co: Getting started with security
• Elastic.co: Java Client and Security
• Elastic.co: Configuring X-Pack Java Clients

The above links refer to the 5.5 version as this is the version supported by Spring Data ElasticSearch at the time of my answer. To summarize those links, here are the steps you have to follow:

1. Include the org.elasticsearch.client:x-pack-transport dependency in your project. You might need to add this repository in the repositories section in your build.gradle/pom.xml: https://artifacts.elastic.co/maven as follows:

   repositories { maven { url "https://artifacts.elastic.co/maven" } }

2. Replace your transport client by one using X-Pack:

   import org.elasticsearch.xpack.client.PreBuiltXPackTransportClient;
   ...
   TransportClient client = new PreBuiltXPackTransportClient(Settings.builder()
       .put("cluster.name", "myClusterName")
       .put("xpack.security.user", "transport_client_user:changeme")
       ...
       .build())
   .addTransportAddress(new InetSocketTransportAddress("localhost", 9300))
   .addTransportAddress(new InetSocketTransportAddress("localhost", 9301));
   

Important: in addition to the above steps for HTTP authentication, you should also set up HTTPS connections to make sure the passwords are not sent in clear over the network.

Bandoline

We're indeed using X-Pack but with Elastic cloud now. At the time of the question we where using ES 2.X though. Thank you for the answer! – Seagraves