Config number_of_shards and number_of_replicas in ELK

Question

I keep studying about ELK Stack and ran into a little problem.

I have been reading all the documentation possible and it makes great emphasis on the importance of shards and replicas. But nowhere does it say how to configure the number of each one. I have read some site that says that it is better to leave it in automatic and others that say how to configure it in version 5.8 but that no longer works.

So if someone can explain to me I would be very grateful.

Jessalin

You need to know how to set the primary and replica shard via the api or how to choose a good number for both? – Breastwork

Answer 1

When you create an index, you can configure both values in the settings of that index:

PUT your-index
{
  "settings": {
    "index.number_of_shards": 3,
    "index.number_of_replicas": 1
  }
}

Also note that you can update the settings of an index after its creation, but you can only update the number of replicas and not the number of primary shards:

PUT your-index/_settings
{
  "settings": {
    "index.number_of_replicas": 2
  }
}

As simple as that!

Odele

There is a default template started by .monitoring... by higher priority. you should set "order" : 1 to override. refer to discuss.elastic.co/t/permanent-way-of-making-es-replicas-0/… – Postbox

Answer 2

Just a small Add-on to @Val answer, related to primary shards.

Reason that you can't change the primary shards are due to the fact, it will change the way data is split between primary shards and changing them will cause consistent hashing to break, which is very popular technique to horizontally scale and splitting the data technique.

Replicas shards are just copy so you can increase and decrease as it doesn't make any impact on consistent hashing.

If you want to change the primary shards, you have to create a new index and use alias API and Reindex API for efficiently doing it,

Discolor