Using tcmalloc/jemalloc with custom memory pool manager

Question

I would like to use a high performance general purpose allocator like jemalloc/tcmalloc with a memory pool. Is there a guide for doing this? I don't want to use jemalloc/tcmalloc as a drop-in replacement for malloc.

I have memory pool that uses libarena and carves a largish memory area (2GB). I want to create fixed size objects pool like gslice on this arena for say 1 GB. (like the Bonwick slab allocator) and want to use the rest of the memory for variable sized general purpose allocation. I would like to have jemalloc/tcmalloc use this memory area. Is this possible? Any ideas as to how I can proceed? If there is an alternate library, I would like to hear about it as well.

Beverage

Good luck getting this answered, I have also really wanted this answered, please see my #23342087 own question. – Deportation

I wrote an allocator specifically for our app. One thing that turned out to be a real nifty idea was the concept of reducing all possible allocations to a small number of "quantum" sizes. Below 4k = 8 bytes, 4k..16k = 16 byte... up to 64mb max. I ended up with only had about 1000 different sizes all the way up to 64 MB. Wasted ram was low since the size of the wasted space was a function of the size of the allocation. This allowed recycle piles for each size. A free was just a push and an alloc was a pop, unless the pile was empty for that size. – Felony

I don't have an answer for you, but you may find some useful info on this jemalloc thread regarding allocating memory as additional arenas for custom allocators here: canonware.com/pipermail/jemalloc-discuss/2015-January/… – Troopship

Answer 1

This is an old question, but there's finally a positive answer, at least where jemalloc is concerned. Since jemalloc version 4.0.0, you can use mallctl to set hooks for where chunked allocations get memory by using the arena.<i>.chunk_hooks interface. These hooks are well documented in the jemalloc manpage, including a number of cross-references.

I'm unaware as to any similar interface in tcmalloc; I do not use it.

I answered a similar question semi-recently; there may be useful information there, as well.

Molli