atomic<bool> vs bool protected by mutex
Asked Answered
I

1

5

Let's assume we have a memory area where some thread is writing data to. It then turns its attention elsewhere and allows arbitrary other threads to read the data. However, at some point in time, it wants to reuse that memory area and will write to it again.

The writer thread supplies a boolean flag (valid), which indicates that the memory is still valid to read from (i.e. he is not reusing it yet). At some point he will set this flag to false and never set it to true again (it just flips once and that is it).

With sequential consistency, it should be correct to use these two code snippets for the writer and the readers, respectively:

...
valid = false;
<write to shared memory>
...

and

...
<read from shared memory>
if (valid) {
    <be happy and work with data read>
} else {
    <be sad and do something else>
}
...

We obviously need to do something to ensure sequential consistency, namely insert the necessary acquire and release memory barriers. We want the flag to be set to false in the writer thread, before writing any data to the segment. And we want the data to be read from memory in the reader threads before checking valid. The later because we know valid to be monotonic, i.e., if it is still valid after reading, it was valid while reading.

Inserting a full fence between memory access and the access to valid will do the trick. I wonder, however, if making valid an atomic will be enough?

std::atomic<bool> valid = true;

Then

...
valid.store(false); // RELEASE
<write to shared memory>
...

and

...
<read from shared memory>
if (valid.load()) { // ACQUIRE
    <be happy and work with data read>
} else {
    <be sad and do something else>
}
...

It seems that in this scenario, the implied release and acquire operations from using the the atomic store and read work against me. The RELEASE in the writer does not prevent the memory access to be moved up over it (just code from above may not be moved down). And similarly, the ACQUIRE in the readers does not prevent the memory access to be moved down over it (just code from below may not be moved up).

If this is true, to make this scenario work, I would need an ACQUIRE (i.e. a load) in the writer thread as well and a RELEASE (i.e. store) in the reader threads. Alternatively, I could just use a normal boolean flag and protect the write and read access (to it only!) in the threads with a shared mutex. By doing so I would effectively also have both an ACQUIRE and a RELEASE in both threads, separating the valid access from the memory access.

So this would be a very severe difference between atomic<bool> and a regular bool protected by a mutex, is this correct?

Edit: There actually seems to be a difference in what is implied by a load and a store on atomics. The std::atomic of C++11 uses memory_order_seq_cst for both (!), rather than memory_order_acquire and memory_order_release for load and store respectively.

In contrast, the tbb::atomic uses memory_semantics::acquire and memory_semantics::release rather than memory_semantics::full_fence.

So if my understanding is correct, the code would be correct with standard C++11 atomics, but with tbb atomics one would need to add the explicit memory_semantics::full_fence template parameter to both load and store.

Indispensable answered 1/7, 2016 at 13:26 Comment(15)
Why would you read before checking for validity? Seems weird to me...Husbandry
Can you not just get the writer to set valid = true; when it finishes writing and the reader to set valid = false; when it's finished reading?Ugh
@Holt: To not disturb the writer (and lock the whole memory block), the readers do an optimistic read and just discard the data if the writer claimed the memory again.Indispensable
@Galik: This is an unrelated question, since it does something different then described in the (simplified) example. In our scenario, the writer must not be prevented from claiming his memory and we are not interested in diving into a full producer/consumer scenario where writer and readers alternate on the same memory.Indispensable
@Indispensable In our scenario, the writer must not be prevented from claiming his memory is mutually exclusive with we are not interested in diving into a full producer/consumer scenario Pick one. You can't have both if the reader(s) are allowed to read from the same memory the writer is writing to.Fabrikoid
@AndrewHenle: That is why I added the "where writer and readers alternate on the same memory". Of course it is a variant of producer/consumer, but not with any kind of rotation, just one monotonic flip of mode of operation.Indispensable
@Indispensable "where writer and readers alternate on the same memory" How do the multiple threads know to alternate? You can't do that with one monotonic flip. You're trying to solve this in a way that requires two-way information flow but only providing one direction.Fabrikoid
Imagine this: reader checks and sees valid flag, starts reading. Writer sets flag to invalid, writes data, sets flag back to valid. Then the reader checks flag and sees that the flag is valid. But between the reader's two checks the writer rewrote the entire memory area. It doesn't get any better by eliminating one of the reader's checks.Fabrikoid
@AndrewHenle: We do not want to alternate (hence the "we are not interested in") :-) I am sorry if I caused any kind of confusion with my comment towards Galik, just stick to the original description.Indispensable
@AndrewHenle, the question says that the flag only gets flipped once; i.e., it's monotonic. So your scenario is not possible.Leupold
@DanRoche You missed my point. Even with checking the boolean flag before and after reading the memory, consistency isn't assured. So how can removing one of those checks ensure consistency?Fabrikoid
@Indispensable I think your edit should be the answer.Leupold
@DanRoche: Thanks for the reply! I find this oddly surprising that the defaults would be chosen like this for the std::atomic, at least after watching Herb Sutter's talk about atomic<> weapons :-)Indispensable
@DanRoche If the flag is only set once, that implies the memory isn't being reused. If the memory region is not being reused, the entire locking scheme is extraneous. And if it is being reused, the locking scheme as described is not sufficient.Fabrikoid
@AndrewHenle: The example is extremely simplified, please do not try to judge its "value" from what is presented. I stripped it to the bare essentials to discuss what is necessary to make this very simple setting work. The atomics we discuss are necessary to ensure sequential consistency. The whole question was about atomic load/store operations (which I assumed would just acquire/release), vs a full memory fence (like through accessing a normal boolean protected by a mutex). It seems a full fence is necessary here, but this can also be achieved by calling load/store with parameters.Indispensable
H
3

The writer switches the valid flag to false and starts writing data, while a reader might still be reading from it.

The design flaw is in the incorrect assumption that reading from and writing to the same memory area is not a problem as long as the reader checks data validity once it has finished reading.

The C++ standard calls it a data race and it leads to undefined behavior.

A correct solution is to use a std::shared_mutex which manages access to a single writer and multiple readers.

Hallway answered 25/2, 2017 at 21:21 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.