c++11 interprocess atomics and mutexes
Asked Answered
A

2

23

I have a Linux program which spawns several processes (fork) and communicates through POSIX Shared Memory. I'd like to have each process allocate an id (0-255). My intention is to place a bitvector in the shared memory region (initialized to zero) and atomically compare and swap a bit to allocate an id.

Is there a c++11-friendly way to do this? Can I create an atomic bitset? Can I use a mutex across processes? How do I assure that constructors get called once and only once across all processes?

Aliform answered 31/10, 2012 at 15:15 Comment(12)
fork returns a pid of child process to the parent proces, why not just use that instead of generating yet another one? Or I'm misunderstanding your questionLeandra
@aleguna Because I would like a value ranging from 0-255 and when a process leaves this program it should free it's ID to be reused.Aliform
@dschatz: "Because I would like a value ranging from 0-255" That doesn't explain why you need that. Especially when the PID mechanism is so much better.Botulin
@NicolBolas I took my problem and selected the minimal necessary information to ask a question that would allow me to solve my problem. I'm not looking for people to change what the problem is. I could create an arbitrary synchronization problem and ask it with respect to posix shared memory across processes and the question still holds. If it helps you, pretend I asked about a shared data structure and how c++ mutexes or atomics would work with respect to that.Aliform
@dschatz: And thus, you have a classic XY question.Botulin
@NicolBolas You clearly missed the point based on your first question. I'm asking generally how c++11 mutexes and atomics work across processes (Hey, the title matches!). I give a specific example to make it concrete. You then proceed to nitpick on the example. If you don't have an answer, that's fine, but there's no reason to try to move the goalposts to answer my question.Aliform
@dschatz: "I'm asking generally how c++11 mutexes and atomics work across processes (Hey, the title matches!)." Then you should not give a very specific, contrived, and useless example as the first paragraph of a general question. If you had stated it as, "I want to know how C++11 threading primitives work across processes. For example, blah", then it would have been clear that the question was general-purpose. A generic title with a specific internal question is common on SO. So much so that I generally ignore titles once I get into reading the question.Botulin
C++11 does not have inter-process functions. Boost::Interprocess might have some tools for you, to use for this problem.Tsimshian
You can use interprocessmutex from Boost (enter link description here)Glazed
Not really what you're asking, but I'd think you'd want to use a bool instead of bit, since you probably want your compare-and-swap to depend just on that single and not 7 additional values.Historical
Regarding bitsets: if atomic access is needed, it would probably be better to use std::vector<std::atomic_int> with one bit per vector element. That way the individual bits could be accessed independently.Florin
This question isn't very good because it contains too many sub-questions.Florin
B
20

The C++11 threading primitives (mutexes, atomics, etc) are threading primitives. The C++ standard doesn't reference processes, and most of these tools don't interoperate across processes.

The only mention of processes in the standard is in a non-normative notation that says that lock-free atomics are intended to be OK for IPC:

Operations that are lock-free should also be address-free. That is, atomic operations on the same memory location via two different addresses will communicate atomically. The implementation should not depend on any per-process state. This restriction enables communication by memory that is mapped into a process more than once and by memory that is shared between two processes.

Outside of this non-normative notation, the threading primitives are not intended to be a means of achieving inter-process communication. The behavior of such objects when placed in shared memory (aside from lock-free atomics as noted above) is undefined.

Botulin answered 31/10, 2012 at 17:19 Comment(7)
[atomics.lockfree] includes "[Note: Operations that are lock-free should also be address-free. That is, atomic operations on the same memory location via two different addresses will communicate atomically. The implementation should not depend on any per-process state. This restriction enables communication by memory that is mapped into a process more than once and by memory that is shared between two processes. — end note ]"Savor
Dear @Nicol Bolas, your answer is great, but I cannot find anywhere or any books writing about that so I still get some confuses about using C++11 mutex in shared memory. (as I know, pthread mutex is ok if using PTHREAD_PROCESS_SHARED)Crazy
I really wish this was included at references such as en.cppreference.com/w/cpp/thread/mutex 😒Latoya
This answer is completely wrong. Jefrrey Yasskin already pointed at one reason why (std::atomic is in fact perfectly suited for shared memory IPC, and sometimes necessary) in his comment, but his comment is easy to gloss over. Mutexes, too, can be used for shared memory IPC, although it's true that the standard library mutexes are not meant for such usage.Florin
And, just to be clear, the intent of the standard is clearly that atomics should be usable for IPC. Jeffrey's quote from the standard shows that this is true. That said, I don't know if that was in the standard already in C++11.Florin
@user2373145: "the intent of the standard is clearly that atomics should be usable for IPC" If it is "clear", why is it in a notation instead of actual text? The standard never mentions processes in normative text. There's a notation about lock-free being "address free", but the C++ memory and object models themselves recognizes no such concept.Botulin
It's "clear intent" because it's right there in the standard, no need to read between the lines. It's only intent, not officially required, because it's not normative. ISO C++ doesn't standardize a process model so they couldn't do that. But all mainstream implementations have lock-free std::atomic that does Just Work in shared memory. (C++20 .wait() and .notify() work thanks to an intentional choice to use shared instead of private futex calls on Linux, but the rest of std::atomic just trivially works because CPUs don't care whether it's a process or a thread.)Gyimah
O
10

You can use mutex inside of shared memory block, but the mutex must be declared as SHARED, therefore is not unusual using mutexes inside of share memory, u can make own class, it is very simple:

class Mutex {
private:
    void *_handle;
public:
    Mutex(void *shmMemMutex,  bool recursive =false, );
    virtual ~Mutex();

    void lock();
    void unlock();
    bool tryLock();
};

Mutex::Mutex(void *shmMemMutex, bool recursive)
{
    _handle = shmMemMutex;
    pthread_mutexattr_t attr;
    ::pthread_mutexattr_init(&attr);
    ::pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
    ::pthread_mutexattr_settype(&attr, recursive ? PTHREAD_MUTEX_RECURSIVE_NP : PTHREAD_MUTEX_FAST_NP);

    if (::pthread_mutex_init((pthread_mutex_t*)_handle, &attr) == -1) {
        ::free(_handle);
        throw ThreadException("Unable to create mutex");
    }
}
Mutex::~Mutex()
{
    ::pthread_mutex_destroy((pthread_mutex_t*)_handle);
}
void Mutex::lock()
{
    if (::pthread_mutex_lock((pthread_mutex_t*)_handle) != 0) {
        throw ThreadException("Unable to lock mutex");
    }
}
void Mutex::unlock()
{
    if (::pthread_mutex_unlock((pthread_mutex_t*)_handle) != 0) {
        throw ThreadException("Unable to unlock mutex");
    }
}
bool Mutex::tryLock()
{
    int tryResult = ::pthread_mutex_trylock((pthread_mutex_t*)_handle);
    if (tryResult != 0) {
        if (EBUSY == tryResult) return false;
        throw ThreadException("Unable to lock mutex");
    }
    return true;
}
Ordzhonikidze answered 11/11, 2013 at 14:35 Comment(3)
Does this really need a virtual destructor?? Also, probably best to use pthread_mutex_t *handle instead of casting it every time you use it.Gyimah
@PeterCordes A virtual dtor isn't necessary, but it does provide a measure of protection if this class is ever subclassed. See localcoder.net/… for details. Should they be used everywhere? That's up to the individual programmer/team to decide. Personally, I don't use them a whole lot, but there are times and places where they absolutely make sense. It can be argued that if you're deleting things enough that the extra cost of calling virtual dtorx is a performance issue, it's almost certainly time to rethink your design.Ljoka
@dgnuff: The vtable pointer wastes space in every instance of the object, so less useful stuff fits in caches. That hurts some even if you never delete these objects. It just seems like pointless waste; you can change it if/when you do want polymorphism.Gyimah

© 2022 - 2024 — McMap. All rights reserved.