ELF64/x86_64 and start address of the memory mapping segment (for shared objects)

Question

I have written several program and found out that when compiled in 64bit, the memory mapping segment (where for example shared objects and shared memory are kept) is always located somewhere around 7f9aca84a000-7fff88400000 but never exactly the same.

I would like to know if there is a fixed start address for this memory segment on x86_64 architecture (ELF64) or what is the maximum and minimum range for this segment?

Here is why I ask this question. We are migrating a system from Tru64 UNIX to Linux. This system used a complex fixed memory mapping of IPC Sys V shared memory, and it is using chained list to go from structure to another inside this segment. With the size and complexity of this piece of code, and the limited time we have at hand, we are trying to find a robust way to fix the start of the share memory (effectively using shmat with a specified address at which to attach the segment). With 64 bit, the virtual address space is so huge (48bit effectively possible addresses) that choosing a "safe" fixed address is much easier and less risky than on 32bit.

Orcein

Answer 1

The x86-64 memory mapping layout is defined in arch/x86/mm/mmap.c. As you can see, there are two strategies used: top-down and bottom-up.

Top-down allocation is the default. It starts at 128MB below the maximum extent of the stack (as defined by the stack rlimit), tweaked by a random offset, and then allocates subsequent mappings downwards in memory from there.

Bottom-up allocation is the fallback. It is used if either:

• The stack rlimit is unlimited;
• The process has the ADDR_COMPAT_LAYOUT personality set; or
• The vm.legacy_va_layout sysctl is non-zero.

Under bottom-up allocation, mapped regions are allocated progressively higher addresses, starting at TASK_SIZE / 3, tweaked by a random offset. TASK_SIZE on x86-64 is 0x800000000000, so bottom-up allocations will start at around 0x2AAAAAAAAAAA.

I would suggest a suitable hole for your fixed mappings that should be OK under either allocation strategy is around 2 * TASK_SIZE / 3 - I would use 0x500000000000.

Levy

Thank you for such an insight. I'm going to look around at all the information you pointed me. – Orcein

so can cant give our own starting address? – Surrebuttal

Answer 2

Is there a fixed start address for mmaped segments?

No. Linux supports ASLR (Address Space Layout Randomization), which means that addresses in programs have some element of randomness. This is to make some exploits less likely to succeed. Furthermore, probably some kernel patches implement different ASLR strategies (they do for regular x86), think of PaX, exec-shield, ...

So, if you want to use a fixed address you may get away with using MAP_FIXED, as @Ethereal recommended.

Agglutinate

Answer 3

I don't have a direct answer for you (yet!), but I can say that I've had success mapping memory well outside of the higher memory ranges. For example:

#include <stdio.h>
#include <sys/mman.h>
#include <stdint.h>

#define ADDRESS 0x700000000

int main(int argc, char *argv[]) {
    uint64_t *map = mmap((void *)ADDRESS, 4096, PROT_READ | PROT_WRITE, MAP_FIXED | MAP_PRIVATE | MAP_ANONYMOUS, 0, 0);

    map[0] = 64;

    printf("Value: %lu\n", map[0]);

    munmap(map, 4096);

    return 0;
}

I'm afraid I don't have time to look through the kernel source at the moment, but I will definitely take a look later. I've always wondered what the answer for this question was . . .

Pounds

Thanks for the information, a similar technique is used in the code with System V IPC, we set a special address when calling shmat. See the man page extract: If shmaddr isn't NULL and SHM_RND is specified in shmflg, the attach occurs at the address equal to shmaddr rounded down to the nearest multiple of SHMLBA. Otherwise shmaddr must be a page-aligned address at which the attach occurs. – Orcein

The difficulty is to determine a proper address, like the one you propose: 0x700000000. And we would like to be able to clearly justify our choice. – Orcein