Solving Redis warnings on overcommit_memory and Transparent Huge Pages for Ubuntu 16.04 on EC2

Question

On a fresh Ubuntu 16.04 EC2 instance the warnings appear like so:

WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.

How to eliminate them permanently?

Tammeratammi

Somehow nobody ventured to question bullshitness of the original premises. vm.overcommit_memory = 0 is a heuristic default (it does overcommit too) and I'd rather rely on kernel devs as opposed to clunky redis team. – Lumbar

Answer 1

Like the warning suggests, just add the line vm.overcommit_memory=1 to the bottom of /etc/sysctl.conf, with something like sudo vi /etc/sysctl.conf.

But permissions don't allow you to edit THP as the warning suggests, so instead do

sudo apt install hugepages

and add the command sudo hugeadm --thp-never to the bottom of your .bashrc, with something like sudo vi ~/.bashrc.

Then just sudo reboot and next time you SSH in run redis-server and the warnings are gone!

Tammeratammi

Is simply making the warning go away dangerous at all? When would it be desirable for a 'background save' to fail under low memory conditions? Could it be problematic to just blindly set 'vm.overcommit_memory=1'? – Fortuity

@ScottSkiles: I believe Redis does a background save by calling fork() which immediately doubles the apparent memory use. But since it is all virtual memory until things change in the parent, things should be OK until the save finishes. – Krafftebing

how to do that in docker container? – Epithet

@HerilMuratovic according to discussions I've seen on github, these settings (probably) go on the host system and docker gets them from there. E.g.: github.com/docker-library/redis/issues/19#issuecomment-96080885 – Alcoholic

Answer 2

With Linux and Ubuntu, I did this before running redis-server, and it seems to work...

sudo add-apt-repository ppa:redislabs/redis
sudo apt-get update
sudo apt-get install redis

https://redis.io/download

Extinguish

Nice, it appears to be a better fix (achieved with the native library). If someone can verify that this solves the problem, then I'll award this as the new best answer. – Tammeratammi

In my case, upgrading redis to v.6.2.6 via the indicated PPA removed the Transparent Huge Page warning, but NOT the overcommit_memory one – Monroy