Elasticsearch cluster connection

Question

It might be a dumb question but i was unable to find answer. If I am having 3 nodes in my cluster so do need to provide the IP and Port of each node while creating transport client so that i can communicate with each node??

new PreBuiltTransportClient(settings, AuthenticationPlugin.class).addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("localhost"),
                        Integer.parseInt("9300")))
                        .addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("localhost"),
                        Integer.parseInt("9301")))
InetSocketTransportAddress(InetAddress.getByName("localhost"),
                        Integer.parseInt("9302")));;

Is there any way where i don't need to provide IP and Port of each node ?? Please help

Commonalty

Answer 1

why not config a FQDN for these nodes, like:

upstream elasticsearch_cluster {
    server 192.168.0.1:9200;
    server 192.168.0.2:9200;
    server 192.168.0.3:9200;
    server 192.168.0.4:9200;
    server 192.168.0.5:9200;
    server 192.168.0.6:9200;
}

server {
    listen 80;
    server_name elasticsearch.example.com;

    location / {
        proxy_pass http://elasticsearch_cluster;
    }
}

Then use elasticsearch.example.com to connect to all nodes.

Guidepost

Answer 2

If all three nodes are in a single cluster, you may communicate only with one of them. They will do all necessary communications behind the scene.

You can also setup cluster to have loadbalancer node without data and always connect to this node. more details here

Update:

Let say you want to run 3 nodes of the same cluster on the same server: node1.local node2.local node3.local

Config files will look like this

node1.local

cluster.name: BillyMiligan
node.name: "node1.local"
network.host: "localhost"
transport.tcp.port: 9301
http.port: 9201
discovery.zen.ping.unicast.hosts: ["node1.local:9301", "node2.local:9302", "node3.local:9303"]
discovery.zen.minimum_master_nodes: 2

node2.local

cluster.name: BillyMiligan
node.name: "node2.local"
network.host: "localhost"
transport.tcp.port: 9302
http.port: 9202
discovery.zen.ping.unicast.hosts: ["node1.local:9301", "node2.local:9302", "node3.local:9303"]
discovery.zen.minimum_master_nodes: 2

node3.local

cluster.name: BillyMiligan
node.name: "node3.local"
network.host: "localhost"
transport.tcp.port: 9303
http.port: 9203
discovery.zen.ping.unicast.hosts: ["node1.local:9301", "node2.local:9302", "node3.local:9303"]
discovery.zen.minimum_master_nodes: 2

Hoopen

If i am connecting to node-1(localhost, 9300) but some how this ES node goes down and there 2 nodes still running on port [(localhost, 9301) and (localhost, 9302) . all nodes are in single cluster then my transport client will automatically get connected to one of node which is running on (9301 OR 9302) ?? – Commonalty

each elasticsearch node uses at least 2 ports (transport.tcp.port: 9300 http.port: 9200 by default) Make sure you override both of them if you starting several nodes on same server – Hoopen

Agreed elasticsearch uses 2 ports for each 1 for tcp and 1 for http, but still I am not clear about my question If i am connecting to node-1(localhost, 9300) but some how this ES node goes down and there 2 nodes still running on port [(localhost, 9301) and (localhost, 9302) . all nodes are in single cluster then my transport client will automatically get connected to one of node which is running on (9301 OR 9302) ?? – Commonalty

I am afraid you are using ports which already taken. By default elasticsearch listen to port 9200 for http request and port 9300 for nodes coordination. You can not run 2 nodes on same server listening to port 9300. you have to override 9200 and 9300 ports. Will add instructions to my answer. – Hoopen

I think there is some confusion, let me explain it one more time. I am having three physical server in same domain, consider 10.0.0.1, 10.0.0.2 and 10.0.0.3. on each each server elasticsearch is running with port. on server 10.0.0.1(ES :- tcp port=9300, http port=9200) on server 10.0.0.2(ES :- tcp port=9300, http port=9200) on server 10.0.0.3(ES :- tcp port=9300, http port=9200) all three ES nodes are in cluster. through java client i am connecting to server 10.0.0.1(ES :- tcp port=9300, http port=9200) using transport client like this – Commonalty