max file descriptors for elasticsearch process is too low

Question

Cannot resolve these problems:

[2017-10-16T13:54:23,381][WARN ][o.e.b.BootstrapChecks    ] [node-1] max 
file descriptors [65000] for elasticsearch process is too low, increase to 
at least [65536]
[2017-10-16T13:54:23,382][WARN ][o.e.b.BootstrapChecks    ] [node-1] max 
number of threads [1024] for user [appadm01] is too low, increase to at 
least [2048]
[2017-10-16T13:54:23,382][WARN ][o.e.b.BootstrapChecks    ] [node-1] system 
call filters failed to install; check the logs and fix your configuration or 
disable system call filters at your own risk

I executed sysctl -w fs.file-max=65536 but I get the same thing.

Thermy

Answer 1

For those who are using docker for running elastic search then you can set ulimit through your docker run command like below.

docker run --ulimit nofile=65536:65536 -p 9200:9200 --name elastic-search docker.elastic.co/elasticsearch/elasticsearch:6.4.2

Hope this will help docker users

Unintelligible

Answer 2

Try this: (got it from 2-3 different sources)

add in $ES_HOME/config/elasticsearch.yml

network.host: 0.0.0.0
http.port: 9200
transport.host: localhost
transport.tcp.port: 9300

If its local dev setup, also use lesser memory options if possible:

add/modify in $ES_HOME/config/jvm.options

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms512m
-Xmx512m

Flak

Thanks, This is worked for me, initialy I have added only "network.host: 0.0.0.0". After adding all above mention, it working without any issue. Thanks lot. – Lucillalucille

in ES 8.8.2 the settin is now "transport.port: 9300". With this small difference , it still worked. – Inexpiable

Answer 3

Just create a systemd service like this, watch closely the ulimit nofile settings : vi /etc/systemd/system/elasticsearch.service :

[Unit]
Description=Spin ES Service
After=network.target

[Service]
User=<user name>
Group=<group name>
Type=simple
ExecStart=/bin/bash /home/<user>/elasticsearch-6.4.3/bin/elasticsearch
Restart=on-failure
RestartSec=5s
LimitAS=infinity
LimitRSS=infinity
LimitCORE=infinity
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

where the ExecStart needs to have "which bash" followed by your $ES_HOME/bin/elasticsearch

Also, my settings for users are these :

ulimit -n 65536

/etc/sysctl.conf

fs.file-max = 2097152
vm.max_map_count = 262144
vm.swappiness = 1

followed by:

sudo sysctl -p

Stucco

Answer 4

Simply Update /etc/security/limits.conf content to below

elasticsearch   soft    nofile          65536
elasticsearch   hard    nofile          65536
elasticsearch   memlock unlimited

and remove content of /etc/security/limits.d/nproc.conf

Do not forgot to relogin into the shell in order to get the limits applied.

Colner

Answer 5

I was also facing the same issue while configuring the sonarqube and was able to resolve the issue by adding the LimitNOFILE=65536 param in the /etc/systemd/system/sonarqube.service file

[Unit]
Description=SonarQube service
After=syslog.target network.target

[Service]
Type=forking

ExecStart=/opt/sonarqube/sonarqube-7.9.1/bin/linux-x86-64/sonar.sh start
ExecStop=/opt/sonarqube/sonarqube-7.9.1/bin/linux-x86-64/sonar.sh stop

User=sonarqube
Group=sonarqube
**LimitNOFILE=65536**
Restart=always

[Install]
WantedBy=multi-user.target

Arterial

Answer 6

I tried this, It worked for me

sudo sysctl -w vm.max_map_count=262144

Randeerandel

What about max file descriptors? – Hilde

Answer 7

This guide will help if you run SonarQube: SonarQube Guide

Sweetie

Answer 8

Worked for me .

If you want to increase the limit shown by ulimit -n, modify below files.

1. /etc/systemd/user.conf

2. /etc/systemd/system.conf

in both file add the following line in addition

  DefaultLimitNOFILE=65536

3. /etc/security/limits.conf with the following lines

            *  soft    nofile  65536
            *  hard    nofile  65536
            elasticsearch   soft    nofile  65536
            elasticsearch   hard    nofile  65536
            elasticsearch   memlock unlimited
   

Colporteur

Answer 9

it works for me after I do: vi /etc/security/limits.conf add following:

• soft nproc 65535
• hard nproc 65535
• soft nofile 65535
• hard nofile 65535

Save and reboot.

Speer

Answer 10

I'm using elasticsearch-7.5.1 on CentOS 7. In my case I needed to edit beyond 65536 and this solved my problem:

Edit the files:

• /etc/security/limits.d/30-elastic.conf

  elasticsearch - nofile 99999
  elasticsearch - noproc 4096 
  

• /usr/lib/systemd/system/elasticsearch.service

  LimitNOFILE=99999
  

Restart the service and check the max_file_descriptors configured for each node using the Nodes stats API, with:

GET _nodes/stats/process?filter_path=**.max_file_descriptors

Gargan

Answer 11

Did you read that page?

It says:

set ulimit -n 65536 as root before starting Elasticsearch, or set nofile to 65536 in /etc/security/limits.conf.

Quenchless

Even after updating ulimit it results the same. max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536] – Yippee

You likely need to log out and relogin into the shell of the runner user for the change to take effect. Can verify by executing ulimit -n. – Threatt

ES is switching versions so fast that above answer for given version and docs may not match. its specially issue in low memory VMs/containers. – Flak