mprotect on a mmap-ed shared memory segment

Question

When two processes share a segment of memory opened with shm_open and then it gets mmap-ed, does doing an mprotect on a portion of the shared memory in one process affects the permissions seen by the other process on this same portion? In other words, if one process makes part of the shared memory segment read-only, does it become read-only for the other process too?

Phonemics

I have found #7217370, but still it would be worth if someone can provide a detailed reply. – Outfox

Answer 1

I always like to address those questions in two parts.

Part 1 - Let's test it

Let's consider an example that is relatively similar to the one at shm_open(3).

Shared header - shared.h

#include <sys/mman.h>
#include <fcntl.h>
#include <semaphore.h>
#include <sys/stat.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

#define errExit(msg) do { perror(msg); exit(EXIT_FAILURE); } while (0)

struct shmbuf {
    char   buf[4096];
    sem_t  sem;
};

Creator process - creator.c (Compile using gcc creator.c -o creator -lrt -lpthread)

#include <ctype.h>
#include <string.h>
#include "shared.h"

int
main(int argc, char *argv[])
{
    if (argc != 2) {
        fprintf(stderr, "Usage: %s /shm-path\n", argv[0]);
        exit(EXIT_FAILURE);
    }

    char *shmpath = argv[1];
    int fd = shm_open(shmpath, O_CREAT | O_EXCL | O_RDWR,
                      S_IRUSR | S_IWUSR);
    if (fd == -1)
        errExit("shm_open");

    struct shmbuf *shm;
    if (ftruncate(fd, sizeof(*shm)) == -1)
        errExit("ftruncate");

    shm = mmap(NULL, sizeof(*shm), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
    if (shm == MAP_FAILED)
        errExit("mmap");

    if (sem_init(&shm->sem, 1, 0) == -1)
        errExit("sem_init");

    if (mprotect(&shm->buf, sizeof(shm->buf), PROT_READ) == -1)
        errExit("mprotect");

    if (sem_wait(&shm->sem) == -1)
        errExit("sem_wait");

    printf("got: %s\n", shm->buf);

    shm_unlink(shmpath);

    exit(EXIT_SUCCESS);
}

Writer process - writer.c (Compile using gcc writer.c -o writer -lrt -lpthread)

#include <string.h>
#include "shared.h"

int
main(int argc, char *argv[])
{
    if (argc != 3) {
        fprintf(stderr, "Usage: %s /shm-path string\n", argv[0]);
        exit(EXIT_FAILURE);
    }

    char *shmpath = argv[1];
    char *string = argv[2];

    int fd = shm_open(shmpath, O_RDWR, 0);
    if (fd == -1)
        errExit("shm_open");

    struct shmbuf *shm = mmap(NULL, sizeof(*shm), PROT_READ | PROT_WRITE,
                              MAP_SHARED, fd, 0);
    if (shm == MAP_FAILED)
        errExit("mmap");

    strcpy(&shm->buf[0], string);

    if (sem_post(&shm->sem) == -1)
        errExit("sem_post");

    exit(EXIT_SUCCESS);
}

What's supposed to happen? The creator process, creates a new shared memory object, "sets" its size, maps it into memory (as shm), uses mprotect to allow only writes to the buffer (shm->buf) and waits for a semaphore to know when the writer (which we will discuss in a moment finishes its thing).

The writer process starts, opens the same shared memory object, writes into it whatever we tell it to and signals the semaphore.

Question is, will the writer be able to write to the shared memory object even though the creator changed the protection to READ-ONLY?

Let's find out. We can run it using:

# ./creator.c /shmulik &
# ./writer.c /shmulik hi!
got: hi!
#
[1]+  Done                    ./creator /shmulik

As you can see, the writer was able to write to the shared memory, even though the creator set it's protection to READ-ONLY.

Maybe the creator does something wrong? Let's try to add the following line to creator.c:

    if (mprotect(&shm->buf, sizeof(shm->buf), PROT_READ) == -1)
        errExit("mprotect");

    memset(&shm->buf, 0, sizeof(shm->buf)); // <-- This is the new line

    if (sem_wait(&shm->sem) == -1)
        errExit("sem_wait");

Let's recompile & run the creator again:

# gcc creator.c -o creator -lrt -lpthread
# ./creator /shmulik
Segmentation fault

As you can see, the mprotect worked as expected.

How about we let the writer map the shared memory, then we change the protection? Well, it ain't going to change anything. mprotect ONLY affects the memory protection of the process calling it (and it's descendants).

Part 2 - Let's understand it

First, you have to understand that shm_open is a glibc method, it's not a systemcall. You can get the glibc source code from their website and just look for shm_open to see that yourself.

The underlying implementation of shm_open is a regular call for open, just like the man page suggests.

As we already saw, most of the magic happens in mmap. When calling mmap, we have to use MAP_SHARED (rather than MAP_PRIVATE), otherwise every process is going to get a private memory segment to begin with, and obviously one ain't gonna affect the other.

When we call mmap, the hops are roughly:

• ksys_mmap_pgoff.
• vm_mmap_pgoff.
• do_mmap
• mmap_region

At that last point, you could see that we take the process' memory management context mm and allocate a new virtual memory area vma:

struct mm_struct *mm = current->mm;
...
vma = vm_area_alloc(mm);
...
vma->vm_page_prot = vm_get_page_prot(vm_flags);

This memory area is not shared with other processes. Since mprotect changes only the vm_page_prot on the per-process vma, it doesn't affect other processes that map the same memory space.

Karolinekaroly

Answer 2

The POSIX specification for mprotect() suggests that changes in the protection of shared memory should affect all processes using that shared memory.

Two of the error conditions detailed are:

• [EAGAIN] The prot argument specifies PROT_WRITE over a MAP_PRIVATE mapping and there are insufficient memory resources to reserve for locking the private page.
• [ENOMEM] The prot argument specifies PROT_WRITE on a MAP_PRIVATE mapping, and it would require more space than the system is able to supply for locking the private pages, if required.

These strongly suggest that memory that is mapped with MAP_SHARED should not fail because of a lack of memory for making copies.

See also the POSIX specification for mmap().

Lavella

Thank you so much for this! Both of the answers are amazing, but I found the one by Daniel a bit more detailed and interesting to follow for unexperienced users like me. Regardless, I insist, thank you, Jonathan. – Outfox

Given a choice between the two answers, I’d accept the other too. – Lavella