How to protect a global variable shared by isr and regular function?

Question

Let's say I have function 1 and an isr routine, both share and update the same flag without any lock between them. the system is single threaded.

the while will be a 3 arm assembly instructions, which means it is not atomic operation, is it ok to share a global variable between non isr and isr functions without any lock or protection?

function 1:

while (flag == false);
flag = false;

isr routine:

do something
flag=true

---

I don't remember there is a linux kernel mechanism for locking between sleepable and non sleepable context e.g. irq and kernel thread.

---

Thanks @artless for his answer here are some issues I am not sure about:

1. Is there a way I won't miss interrupt at all ?

2. How the memory barriers solve the issue, does it have effect when the code runs on single cpu ?

3. What is the expected behavior when using barriers between different contexts ?

4. Can a sleep in the while loop can solve problems of syncs ?

Sixtyfourmo

Make sure that flag is declared volatile, otherwise the compiler is free to optimize while (flag == false); as while (true); since it will likely detect that flag cannot be changed from within function 1. Marking it volatile informs the compiler that the variable is subject to change at any time. (Since the system is single threaded, a memory barrier should not be required.) – Precipitancy

Answer 1

Using volatile is often quoted as a solution, but this is not quite true. It will often mask a problem as volatile will always make code slower. If your only use is as shown, then volatile will probably work.

It is probably better with a single reader and single write to use memory barriers. This would be your code then,

Mainline:

volatile int *p = &flag;
while (*p == false);   /* You must use volatile if you poll */
flag = false;
asm volatile ("" : : : "memory"); /* gcc barrier */

isr:

/* do something */
flag=true
asm volatile ("" : : : "memory"); /* gcc barrier */

Here, the barrier just forces the compiler to do the ARM str instruction at that point. The optimizer will not move any code before or after. You can also use swp or ldrex and strex depending on your ARM CPU. As well, ring buffers are often used with ISR and mainlines as they don't need any special CPU support; only the compiler memory barrier.

See the lock-free and specifically search lock-free and arm.

Edit: For additions,

Is there a way I won't miss interrupt at all ?

This is dependent on the interrupt source. If it is a timer and you know the timer source can never be faster than XX instructions and no other interrupts are active in the system, then your current code will work. However, if the interrupt is from an external source like an Ethernet controller, a non-debounced keypad, etc. It is possible for multiple interrupts to come quickly. Some times new interrupts even happen during the interrupt handler. Depending on the ISR source, there are different solutions. A ring buffer is commonly used to queue work items from the ISR for the main line. For a UART, the ring might contain actual character data. It could be a list of pointer, etc. It is difficult to synchronize the ISR from the mainline when the communication becomes more complex; So I believe the answer depends on the interrupt source. This is why every OS has so many primitives and infra-structure for this issue.

How the memory barriers solve the issue, does it have effect when the code runs on single cpu ?

Memory barriers don't completely solve the missed interrupt issue; just like volatile doesn't. They just make the window much smaller. They force the compiler to schedule a load or store earlier. For example the main line loop,

  1: ldr r0, [r1]
     cmp r0, #0    ; xxx
     bne 1b        ; xxx
     mov r0,#1     ; xxx
     str r0, [r1]

If a 2nd interrupt happens during the xxx lines, then your flag should be set twice and you missed one interrupt. The barriers just make sure the compiler places the ldr and str close together.

What is the expected behavior when using barriers between different contexts?

The compiler memory barrier I show just makes the compiler do stuff sooner. It has no effect between contexts. There are different barriers; but mostly they are for multi-CPU designs.

Can a sleep in the while loop can solve problems of syncs?

Not really, this is just a more efficient use. The ARM WFI instruction can temporarily stop the CPU and this will save power. That is normally what sleep() does on the ARM. I think you need to change the communication between the ISR and the mainline, if this is an issue. That depends on the ISR source.

Chieftain

Linux's memory barriers gives a good example of different types of issues you may encounter with synchronization. Your current structure can always miss an interrupt, for instance. Ie, between polling and resetting the flag to false, another interrupt may occur. – Chieftain

Updated my question, and found nice reference about arm and memory: engenuics.com/mpg/notes/notes_mpgl1_chapter6.pdf – Sixtyfourmo

thanks again for your awesome answers. So basically there is no 100% method I can use and won't miss any interrupt ? the interrupt source could be UART/USB, which is external, can you consult what mechanism can I consider for that? – Sixtyfourmo

There are 100% ways for different devices. I don't think that this flag method is reliable for all devices. You need to service the device in the ISR and then queue work to a mainline. Your flag method works pretty well with a timer tick interrupt for instance. – Chieftain

Is volatile still needed if memory barriers are used and mainline only access the data with interrupts disabled? This was asked here but there was no solid answer. – Prattle

Generally volatile is not needed/desired if you use a compiler memory barrier. Whether it is needed or not is a question of how the variables are being used. Mainline read/clear, interrupt set? Or is there calculation going on? Is the memory cached/non-cached? Is it SMP? If things seem to correct by adding volatile to a memory barrier, you probably just changed the timing and a race still exists. – Chieftain

It is more portable to use the C11 and C++11 atomic APIs. These will provide the necessary memory barriers without resorting to platform dependent code. – Floating

That is true for typical user space programming. It may not be true for functionally safe programming where only certain certified tools and libraries maybe used. So a good point, but not 100% universally true; especially in the context of system level driver code. I think standard atomics do very well for SMP synchronization, but often there are OS level nuances where they do not do as well for ISRs. Absolutely worth considering C11/C++ atomics. Platform dependant code can be localized and well documented; so it should be balanced with other factors. Answered in spirit of the question. – Chieftain

Answer 2

It will be better if you can declare the flag as : volatile int flag; or volatile bool flag;

Cabalism

Answer 3

This should help prevent missed interrupt. It is based on the wonderfully detailed ans from @artless_noise

Here the ISR posts on a semaphore(non blocking call). Adding the barrier to ensure that write completes. The thread will run as many times as the semaphore is posted.

sem = sem_open(argv[optind], flags, perms, 0); // Initialising semaphore to 0

function 1:

while(sem_getvalue(sem) > 0)
{
   flag = false;
   //Avoiding the barrier if this value isnt needed by ISR
   asm volatile ("" : : : "memory"); /* gcc barrier */
   //perform action
}

isr routine:
    do something
    flag=true;
    asm volatile ("" : : : "memory"); /* gcc barrier */
    sem_post(sem);

Naevus

Answer 4

Yes. If it is a single threaded model, there is no need for a lock.

Touraco