Rabbitmq File Descriptor Limit

Question

Rabbitmq documentation says that we need to do some configuration before we use it on production. One of the configuration is about maximum open file number (which is an OS parameter).

Rabbitmq server we use is running on Ubuntu 16.04 and according to resources I found on web, I updated the number of open files as 500k. When I check it from command line, I get the following output:

root@madeleine:~# ulimit -n
500000

However when I look at the rabbitmq server status, I see another number.

root@madeleine:~# rabbitmqctl status | grep 'file_descriptors' -A 4
 {file_descriptors,
     [{total_limit,924},
      {total_used,19},
      {sockets_limit,829},
      {sockets_used,10}]},

It seems like, I managed to increase the limit on OS side, but rabbitmq still thinks that total limit of file descriptors is 924.

What might be causing this problem?

Bland

That depends entirely on how you're running RabbitMQ, and how you configured the open file limit. – Bummer

@RogerLipscombe I am using a config file to run Rabbitmq. Its content is here, can you take a look at it? [{rabbit, [{vm_memory_high_watermark, 0.4},{disk_free_limit, {mem_relative, 2.0}}]}]. – Bland

Which OS are you using? do you use systemd? – Flatways

Ubuntu 16.04.2 LTS and yes I am using systemd – Bland

Answer 1

You might want to look at this page

Apparently, this operation depends on the OS version. If you have a systemd, you should do the following in /etc/systemd/system/rabbitmq-server.service.d/limits.conf file:

Notice that this service configuration might be somewhere else according to the operating system you are using. You can use the following command to find where this service configuration is located and update that file.

find / -name "*rabbitmq-server.service*"

[Service]

LimitNOFILE=300000

On the other hand, if you do not have the systemd folder, you should try this in your rabbitmq-env.conf file:

ulimit -S -n 4096

Foreignborn

ubuntu 16.04 /lib/systemd/system/rabbitmq-server.service add LimitNOFILE under [Service] block works – Viewpoint

@cwhsu, this is really bad advice as that file can be overwritten by package updates. – Stead

The proper way to edit systemd service is sudo systemctl edit rabbitmq-server.service - this will figure out what is the correct location for your override file and open it in your preferred editor. Then you can just add the [Service] block as suggested, save and quit. systemctl will then also reload the configuration for you. – Stead

Another way (ubuntu 16.04). After installation there is no folder /etc/systemd/system/rabbitmq-server.service.d. Just create it and put file limits.confin that folder. In file add the [Service] block as in answer. Then in console sudo service rabbitmq-server restart for restart and sudo rabbitmqctl status for check. – Tremain

Answer 2

Increase / Set maximum number of open files

sudo sysctl -w fs.file-max=65536
These limits are defined in /etc/security/limits.conf

sudo nano /etc/security/limits.conf
and set

soft nofile 65536
hard nofile 65536
Per user settings for rabbitmq process can also be set in /etc/default/rabbitmq-server

sudo nano /etc/default/rabbitmq-server
and set

ulimit -n 65536
Then reboot the server for changes to take effect.

Forereach

These steps seems like working. ulimit -n now returns 65536 but still, in rabbitmqctl status command's output, file_descriptors values are not increased. – Bland