Is `memcpy((void *)dest, src, n)` with a `volatile` array safe?
Asked Answered
B

3

12

I have a buffer that I use for UART, which is declared this way:

union   Eusart_Buff {
    uint8_t     b8[16];
    uint16_t    b9[16];
};

struct  Eusart_Msg {
    uint8_t             msg_posn;
    uint8_t             msg_len;
    union Eusart_Buff   buff;
};

struct  Eusart {
    struct Eusart_Msg   tx;
    struct Eusart_Msg   rx;
};

extern  volatile    struct Eusart   eusart;

And here is the function that fills the buffer (which will be sent using interrupts):

void    eusart_msg_transmit (uint8_t n, void *msg)
{

    if (!n)
        return;

    /*
     * The end of the previous transmission will reset
     * eusart.tx.msg_len (i.e. ISR is off)
     */
    while (eusart.tx.msg_len)
        ;

    if (data_9b) {
        memcpy((void *)eusart.tx.buff.b9, msg,
                sizeof(eusart.tx.buff.b9[0]) * n);
    } else {
        memcpy((void *)eusart.tx.buff.b8, msg,
                sizeof(eusart.tx.buff.b8[0]) * n);
    }
    eusart.tx.msg_len   = n;
    eusart.tx.msg_posn  = 0;

    reg_PIE1_TXIE_write(true);
}

At the moment of using memcpy(), I know no one else is going to use the buffer (atomic), because the while loop ensures that the last message has been sent, and therefore the interrupt is disabled.

Is it safe to cast away volatile this way so that I am able to use memcpy() or should I make a function maybe called memcpy_v() like these to be safe?:

void *memcpy_vin(void *dest, const volatile void *src, size_t n)
{
    const volatile char *src_c  = (const volatile char *)src;
    char *dest_c                = (char *)dest;

    for (size_t i = 0; i < n; i++)
        dest_c[i]   = src_c[i];

    return  dest;
}

volatile void *memcpy_vout(volatile void *dest, const void *src, size_t n)
{
    const char *src_c       = (const char *)src;
    volatile char *dest_c   = (volatile char *)dest;

    for (size_t i = 0; i < n; i++)
        dest_c[i]   = src_c[i];

    return  dest;
}

volatile void *memcpy_v(volatile void *dest, const volatile void *src, size_t n)
{
    const volatile char *src_c  = (const volatile char *)src;
    volatile char *dest_c       = (volatile char *)dest;

    for (size_t i = 0; i < n; i++)
        dest_c[i]   = src_c[i];

    return  dest;
}

Edit:

If I need those new functions, given that I know no one is going to modify the array at the same time, would it make sense to use restrict to (maybe) help the compiler optimize (if it can)? Possibly this way (correct me if I'm wrong):

volatile void *memcpy_v(restrict volatile void *dest,
                        const restrict volatile void *src,
                        size_t n)
{
    const restrict volatile char *src_c = src;
    restrict volatile char *dest_c      = dest;

    for (size_t i = 0; i < n; i++)
        dest_c[i]   = src_c[i];

    return  dest;
}

Edit 2 (add context):

void    eusart_end_transmission (void)
{

    reg_PIE1_TXIE_write(false); /* TXIE is TX interrupt enable */
    eusart.tx.msg_len   = 0;
    eusart.tx.msg_posn  = 0;
}

void    eusart_tx_send_next_c   (void)
{
    uint16_t    tmp;

    if (data_9b) {
        tmp     = eusart.tx.buff.b9[eusart.tx.msg_posn++];
        reg_TXSTA_TX9D_write(tmp >> 8);
        TXREG   = tmp;
    } else {
        TXREG   = eusart.tx.buff.b8[eusart.tx.msg_posn++];
    }
}

void __interrupt()  isr(void)
{

    if (reg_PIR1_TXIF_read()) {
        if (eusart.tx.msg_posn >= eusart.tx.msg_len)
            eusart_end_transmission();
        else
            eusart_tx_send_next_c();
    }
}

Although volatile may not be is needed (I asked it in another question: volatile for variable that is only read in ISR?), this question still should be answered in the assumption that volatile is needed so that future users that really need volatile (for example me when I implement the RX buffer), can know what to do.


EDIT (Related) (Jul/19):

volatile vs memory barrier for interrupts

Basically says that volatile is not needed, and therefore this issue disappears.

Bucentaur answered 2/3, 2019 at 23:38 Comment(10)
Does your platform specify that volatile makes objects thread-safe? Because on most platform, that's not true.Estate
It's thread-safe not because of volatile, but because there is only one thread, and also the interrupt is checked to be disabled before I start to write, and enabled after that. So 0 possibility of someone messing around at the same time.Bucentaur
What do you even need volatile for?Pomeroy
Because the variable is used in normal code and also in an interrupt. It's just that in the moment of writing to it, I certify nobody else is using it, but in any other moment, the variable is shared across the main loop and the interrupt.Bucentaur
So, I write to it atomically, but that is completely different from the concept of volatile, which I also need.Bucentaur
My understanding is that, strictly speaking, if you access a variable that has a volatile qualifier through a non-volatile pointer, you invoke undefined behaviour. Therefore, your use of 'plain' memcpy() is dubious, even if unlikely to actually cause trouble.Rudy
@chux Sorry, I changed the restrict code because it was not the main point of the question, but I'll revert that. Did you mean that part? (Already reverted)Bucentaur
By the way, I already upvoted your answer, because it clearly deserves it. I just didn't mark it as "answers the problem" because it is not yet complete, but soon will.Bucentaur
If you didn't mean that, but the edit about the context, I added that because you answer seemed to be asking for me to do that. I did it so that your answer is complete and not like "posted code is insufficient to be certain". Again sorry.Bucentaur
Yes good to revert back to existing restrict code and append new info.Bruxelles
B
7

Is memcpy((void *)dest, src, n) with a volatile array safe?

No. In the general case, memcpy() is not specified to work correctly with volatile memory.
OP's case looks OK to cast away volatile, yet posted code is insufficient to be certain.

If code wants to memcpy() volatile memory, write the helper function.

OP's code has restrict in the wrong place. Suggest

volatile void *memcpy_v(volatile void *restrict dest,
            const volatile void *restrict src, size_t n) {
    const volatile unsigned char *src_c = src;
    volatile unsigned char *dest_c      = dest;

    while (n > 0) {
        n--;
        dest_c[n] = src_c[n];
    }
    return  dest;
}

A singular reason for writing your own memcpy_v() is the a compiler can "understand"/analyze memcpy() and emit code that is very different than expected - even optimize it out, if the compiler thinks the copy is not needed. Remind oneself that the compiler thinks memcpy() manipulated memory is non-volatile.


Yet OP is using volatile struct Eusart eusart; incorrectly. Access to eusart needs protection that volatile does not provide.

In OP's case, code can drop volatile on the buffers and then use memcpy() just fine.

A remaining issue is in the scant code of how OP is using eusart. Using volatile does not solve OP's problem there. OP's does assert "I write to it atomically,", yet without posted atomic code, that is not certain.


Code like the below benefits with eusart.tx.msg_len being volatile, yet that is not sufficient. volatile insures .tx.msg_len is not cached and instead re-reads each time.

while (eusart.tx.msg_len)
    ;

Yet the read of .tx.msg_len is not specified as atomic. When .tx.msg_len == 256 and a ISR fires, decrementing .tx.msg_len, the read of the the LSbyte (0 from 256) and MSbyte (0 from 255), the non-ISR code may see .tx.msg_len as 0, not 255 nor 256, thus ending the loop at the wrong time. The access of .tx.msg_len needs to be specified as indivisible (atomic), else, once in a while code fails mysteriously.

while (eusart.tx.msg_len); also suffers from being an end-less loop. Should the transmission stop for some reason other than empty, the while loop never exits.

Recommend instead to block interrupts while inspecting or changing eusart.tx.msg_len, eusart.tx.msg_posn. Review your compilers support of atomic or

size_t tx_msg_len(void) {
  // pseudo-code
  interrupt_enable_state = get_state();
  disable_interrupts();
  size_t len = eusart.tx.msg_len;
  restore_state(interrupt_enable_state);
  return len;
}

General communication code ideas:

  1. While non-ISR code reads or writes eusart, make sure the ISR cannot ever change eusart.

  2. Do not block ISR for long in step #1.

  3. Do not assume underlying ISR() will chain input/output successfully without a hiccup. Top level code should be prepared to re-start output if it gets stalled.

Bruxelles answered 3/3, 2019 at 4:58 Comment(17)
I'll add more code so that you have full context :)Bucentaur
Why don't you also put restrict in src_c & dest_c? It isn't needed there, right?Bucentaur
Also, why use a while, when the for is more explicit? Won't the compiler be able to optimize away the index and reverse the order if it saves one instruction?Bucentaur
unsigned is unnecessary: https://mcmap.net/q/753289/-implement-memcpy-is-unsigned-char-needed-or-just-charBucentaur
Thanks. I fixed the usage of restrict.Bucentaur
@CacahueteFrito Re restrict not needed as the compiler, by analyzing *memcpy_v() alone, can deduce the restrict inheritance.Bruxelles
@CacahueteFrito why use a while --> Just another way to do the same thing.Bruxelles
Ok. Now that I posted the isr code and all relevant uses of eusart, could you please update your answer based on that?Bucentaur
@CacahueteFrito SureBruxelles
"When .tx.msg_len == 256 and a ISR fires," It cannot happen, because .tx.msg_len is uint8_t. Although good info for the general case when it may be uint16_tBucentaur
"Should the transmission stop for some reason other than empty, the while loop never exits." In this simple case, this is OK because there is only one place in the code where I stop the transmission, and that is eusart_tx_end_transmission() which does stop the interrupt, and immediately resets .tx.msg_len.Bucentaur
@CacahueteFrito Fair enough about ".tx.msg_len is uint8_t". I had reasoned it as size_t.Bruxelles
By idea.3 you mean there could be a hardware failure that blocked the ISR from firing, but the buffer would still be non-empty, so that the code would be waiting forever?Bucentaur
@CacahueteFrito Yes something like that. Or consider what would happen with while (eusart.tx.msg_len) ; and interrupts were disabled.Bruxelles
If the user (at the moment that's also me) does only use the functions I'm writing, that will never happen. However it is true that it could directly write into the interrupt enable register, which would be fatal. The benefit of not doing that check is that the code is simpler. And as long as the user doesn't do anything really stupid, it should work. The hardware failure would be a problem, though.Bucentaur
The hardware failure would be a problem, though... In that case, if the user knows there is a failure, and calling this function would become a while (1);, it can call eusart_end_transmission() just before, and problem solved. If it doesn't know it, I don't think I will be able to check for that either, so don't care.Bucentaur
As your answer to the other question (https://mcmap.net/q/753287/-volatile-for-variable-that-is-only-read-in-isr) has changed, I think you should update this one accordingly (volatile is needed on the buffer).Bucentaur
U
2

The Standard lacks any means by which programmers can demand that operations that access a region of storage by means of an ordinary pointer are completed before a particular volatile pointer access is performed, and also lacks any means of ensuring that operations which access a region of storage by means of an ordinary pointer are not carried out until after some particular volatile pointer access is performed. Since the semantics of volatile operations are Implementation-Defined, the authors of the Standard may have expected that compiler writers would recognize when their customers might need such semantics, and specify their behavior in a fashion consistent with those needs. Unfortunately, that hasn't happened.

Achieving the semantics you require will either making use of a "popular extension", such as the -fms-volatile mode of clang, a compiler-specific intrinsic, or else replacing memcpy with something that's so horribly inefficient as to swamp any supposed advantage compilers could gain by not supporting such semantics.

Undertone answered 4/3, 2019 at 23:11 Comment(8)
I don't fully understand the first part. Are you saying basically that volatile is not atomic? As I understand volatile, it just means that the compiler isn't allowed to remove code that could seem unnecessary, right?Bucentaur
Also, I've never used clang for something serious (always GCC); could you provide a link for info on that -fms-volatile?Bucentaur
@CacahueteFrito: Some compilers such as the MSVC (Microsoft) compiler will refrain from reordering any operations across volatile operations, and from what I can tell Clang can be configured to behave likewise via the use of the indicated flag. The clang documentation I've seen is a bit vague as to exactly what is and isn't promised, alas.Undertone
By the standard, volatile is guaranteed to follow the order you write it: <C11: 5.1.2.3 Program execution>.Bucentaur
@CacahueteFrito: The Standard only requires that operations on volatile-qualified objects be ordered relative to other operations on volatile-qualified objects. There is no standard means of forcing ordering relative to non-qualified objects..Undertone
What I mean is that volatile seems very well defined in the Standard to me. In this case, I don't even care about the order, as long as it is done, because during the memcpy no one else is using the buffer (and therefore the *restrict).Bucentaur
@CacahueteFrito: The problem is that if one does something like memcpy(buffer, src1, 40); outptr = buffer; outlen = 40; while(outlen) ; memcpy(buffer, src2, 25); outptr = buffer; outlen = 25; while(outlen) ; [with outptr and outlen qualified volatile] there's no way to force the compiler to actually write all 40 bytes of src1 rather than just the portion that won't get overwritten by the contents of src2. On some platforms, there's no way that could matter, but on other platforms the full copy could be vital for correctness.Undertone
Let us continue this discussion in chat.Bucentaur
S
2

The C17 standard says this in section 6.7.3 "Type qualifiers":

If an attempt is made to refer to an object defined with a volatile-qualified type through use of an lvalue with non-volatile-qualified type, the behavior is undefined.135)

135)This applies to those objects that behave as if they were defined with qualified types, even if they are never actually defined as objects in the program (such as an object at a memory-mapped input/output address).

So no, that's not safe.

Sherlocke answered 15/6, 2022 at 21:32 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.