When to turn off Transparent Huge Pages for redis

Question

According to redis docs, it's advisable to disable Transparent Huge Pages.

Would the guidance be the same if the machine was shared between the redis server and the application.

Moreover, for other technologies, I've also read guidance that THP should be disabled for all production environments when setting up the server. Is this kind of pre-emptiveness applicable to redis as well, or one must first strictly monitor latency issues before deciding to turn off THP?

Coagulant

Answer 1

Note: At the time this answer was originally written, turning off THP was still advised all the time. Things have changed since this and it is no longer advisable to turn off THP unless you are persisting to disk. I am only leaving this answer here for posterity's sake.

Turn it off. The problem lies in how THP shifts memory around to try and keep or create contiguous pages. Some applications can tolerate this, most databases cannot and it causes intermittent performance problems, some pretty bad. This is not unique to Redis by any means.

For your application, especially if it is JAVA, set up real HugePages and leave the transparent variety out of it. If you do that just make sure you allocate memory correctly for the app and redis. Though I have to say, I probably would not recommend running both the app and redis on the same instance/server/vm.

Anguish

Well to be exact, I've got a Django (Python) application, although I fathom that won't change anything vis-a-vis your answer, would it? – Coagulant

No, it would not. – Anguish

Thanks Kirk! Btw to further clarify, I need to do both echo never > /sys/kernel/mm/transparent_hugepage/enabled and echo never > /sys/kernel/mm/transparent_hugepage/defrag, correct? – Coagulant

Moreover, in your experience, does req/s throughput suffer as a result of turning off THP? – Coagulant

More than likely, you need both places. You will need to have some kind of init script that starts on boot that will do it every time. – Anguish

Each statement/claim of this answer is incorrect or untrue. That, which is claimed with no evidence, can be dismissed without any evidence. – Important

Answer 2

Turning off transparent hugepages is a bad idea, and redis no longer recommends it.

What you should do instead is make sure transparent_hugepage is not set to always. (This is what recent versions of redis check for.) You can check the current value of the setting with:

$ cat /sys/kernel/mm/transparent_hugepage/enabled

And correct it like so:

# echo madvise >/sys/kernel/mm/transparent_hugepage/enabled

Although no action is likely to be necessary, since madvise is typically the default setting in recent linux distros.

Some background:

• transparent_hugepage = always: can force applications to use hugepages unless they opt out with madvise. This has a lot of problems and is rarely enabled.
• transparent_hugepage = never: does not fulfill allocations with hugepages, even if the application requests it with madvise
• transparent_hugepage = madvise: allows applications to opt-in to hugepages. This is normally a good idea because hugepages can improve performance in some applications, but this setting doesn't force them on applications that, like redis, don't opt in

Quimby

It is still recommended by Redis documentation when using Redis persistence: redis.io/docs/management/optimization/latency/… – Tillford

Answer 3

It is rather annoying that searching for "transparent huge pages" yields top results about how to disable them because Redis and some other databases cannot handle transparent huge pages without performance degradation.

These applications should do any of:

• Use prctl(PR_SET_THP_DISABLE, ...) call to opt out from using transparent huge pages.
• Be started by a script which does this call for them prior to fork/exec the database process. PR_SET_THP_DISABLE get inherited by child processes/threads for exactly this scenario when an existing application cannot be modified.

prctl(PR_SET_THP_DISABLE, ...) has been available since Linux 3.15 circa 2014, so that there is little excuse for those databases to not mention this solution, instead of giving this poor/panic advice to their users to disable transparent huge pages for the entire system.

3 years after this question was asked, Redis got disable-thp config option to make prctl(PR_SET_THP_DISABLE, ...) call on its own, by default.

---

My production memory-intensive processes go 5-15% faster with /sys/kernel/mm/transparent_hugepage/enabled set to always. Many popular desktop applications benefit from always transparent huge pages immensely.

This is why I cannot appreciate those search results for "transparent huge pages" spammed with Redis adviсe to disable them. That's a panic advice from Redis, not the best practice.

Important

Answer 4

The overhead THP imposes occurs only during memory allocation, because of defragmentation costs.

If your redis instance has a (near-)constant memory footprint, you can only benefit from THP. Same applies to java or any other long-lived service that does its own memory management. Pre-allocate memory once and benefit.

Preiser

Answer 5

why playing such echo-games when there is a kernel-param you can boot with?

transparent_hugepage=never

Bogosian

not sure why this answer got downvoted. Here's another reference unix.stackexchange.com/a/99172/29023 – Kurdistan

perhaps you didn't need to call the other answer "echo-games" and show more details about what you wanted to say as the link above – Kurdistan

sorry, but when one is not capable to copy&paste "transparent_hugepage=never" into Google or don't know what a kernel parameter is he's not the audience anyways – Bogosian

Simple. The OP raised a number of concerns and none of them were addressed by this answer. – Insist

i refered to "echo never > /sys/kernel" which is not persistent while "transparent_hugepage=never" as kenerl param is a) persistent and b) as early as possible set – Bogosian

This is to be set in sysctl.conf? – Coagulant

The question in "When", not "How". For recent distro, I think the answer by Kaz Wesley (i.e., do not turn off) is more appropriate. – Spurious