What is the Re-entrant lock and concept in general?

Question

I always get confused. Would someone explain what Reentrant means in different contexts? And why would you want to use reentrant vs. non-reentrant?

Say pthread (posix) locking primitives, are they re-entrant or not? What pitfalls should be avoided when using them?

Is mutex re-entrant?

Dy

Answer 1

Re-entrant locking

A reentrant lock is one where a process can claim the lock multiple times without blocking on itself. It's useful in situations where it's not easy to keep track of whether you've already grabbed a lock. If a lock is non re-entrant you could grab the lock, then block when you go to grab it again, effectively deadlocking your own process.

Reentrancy in general is a property of code where it has no central mutable state that could be corrupted if the code was called while it is executing. Such a call could be made by another thread, or it could be made recursively by an execution path originating from within the code itself.

If the code relies on shared state that could be updated in the middle of its execution it is not re-entrant, at least not if that update could break it.

A use case for re-entrant locking

A (somewhat generic and contrived) example of an application for a re-entrant lock might be:

• You have some computation involving an algorithm that traverses a graph (perhaps with cycles in it). A traversal may visit the same node more than once due to the cycles or due to multiple paths to the same node.

• The data structure is subject to concurrent access and could be updated for some reason, perhaps by another thread. You need to be able to lock individual nodes to deal with potential data corruption due to race conditions. For some reason (perhaps performance) you don't want to globally lock the whole data structure.

• Your computation can't retain complete information on what nodes you've visited, or you're using a data structure that doesn't allow 'have I been here before' questions to be answered quickly.
  
  An example of this situation would be a simple implementation of Dijkstra's algorithm with a priority queue implemented as a binary heap or a breadth-first search using a simple linked list as a queue. In these cases, scanning the queue for existing insertions is O(N) and you may not want to do it on every iteration.

In this situation, keeping track of what locks you've already acquired is expensive. Assuming you want to do the locking at the node level a re-entrant locking mechanism alleviates the need to tell whether you've visited a node before. You can just blindly lock the node, perhaps unlocking it after you pop it off the queue.

Re-entrant mutexes

A simple mutex is not re-entrant as only one thread can be in the critical section at a given time. If you grab the mutex and then try to grab it again a simple mutex doesn't have enough information to tell who was holding it previously. To do this recursively you need a mechanism where each thread had a token so you could tell who had grabbed the mutex. This makes the mutex mechanism somewhat more expensive so you may not want to do it in all situations.

IIRC the POSIX threads API does offer the option of re-entrant and non re-entrant mutexes.

Innermost

Although such situations should usually be avoided anyway, as it makes it hard to avoid deadlock etc as well. Threading is hard enough anyway without being in doubt as to whether you've already got a lock. – Radioman

+1, also consider the case where the lock is NOT reentrant, you can block on yourself if you're not careful. Plus in C, you don't have the same mechanisms other languages do to ensure the lock is Released as many times as it is Acquired. This can lead to big problems. – Stpeter

that's exact ly what happened to me yesterday: I didn't take the issue of re-entrancy into a consideration and ended up debugging a deadlock for 5 hours... – Dy

@Jon Skeet - I think there are probably situations (see my somewhat contrived example above) where keeping track of locks is impractical due to performance or other considerations. – Innermost

Answer 2

A re-entrant lock lets you write a method M that puts a lock on resource A and then call M recursively or from code that already holds a lock on A.

With a non re-entrant lock, you would need 2 versions of M, one that locks and one that doesn't, and additional logic to call the right one.

Bloem

Does this mean if i have recursive calls acquiring same lock obj more than once - say x times by a given thread, i can't interleave the execution without releasing all the recursively acquired locks (same lock but for x number of times)? If true, then it essentially makes this implementation sequential. Am i missing something? – Extensity

That shouldn't be a real worlld problem. It's more about granular locking and that a Thread won't lock itself out. – Bloem

Answer 3

Reentrant lock is very well described in this tutorial.

The example in the tutorial is far less contrived than in the answer about traversing a graph. A reentrant lock is useful in very simple cases.

Scorcher

Answer 4

The what and why of recursive mutex should not be such a complicated thing described in the accepted answer.

I would like to write down my understanding after some digging around the net.

---

First, you should realize that when talking about mutex, multi thread concepts is definitely involved too. (mutex is used for synchronization. I don't need mutex if I only have 1 thread in my program)

---

Secondly, you should know the difference bewteen a normal mutex and a recursive mutex.

Quoted from APUE:

(A recursive mutex is a) A mutex type that allows the same thread to lock it multiple times without first unlocking it.

The key difference is that within the same thread, relock a recursive lock does not lead to deadlock, neither block the thread.

Does this mean that recusive lock never causes deadlock?
No, it can still cause deadlock as normal mutex if you have locked it in one thread without unlocking it, and try to lock it in other threads.

Let's see some code as proof.

1. normal mutex with deadlock

#include <pthread.h>
#include <stdio.h>

pthread_mutex_t lock;


void * func1(void *arg){
    printf("thread1\n");
    pthread_mutex_lock(&lock);
    printf("thread1 hey hey\n");

}


void * func2(void *arg){
    printf("thread2\n");
    pthread_mutex_lock(&lock);
    printf("thread2 hey hey\n");
}

int main(){
    pthread_mutexattr_t lock_attr;
    int error;
//    error = pthread_mutexattr_settype(&lock_attr, PTHREAD_MUTEX_RECURSIVE);
    error = pthread_mutexattr_settype(&lock_attr, PTHREAD_MUTEX_DEFAULT);
    if(error){
        perror(NULL);
    }

    pthread_mutex_init(&lock, &lock_attr);

    pthread_t t1, t2;

    pthread_create(&t1, NULL, func1, NULL);
    pthread_create(&t2, NULL, func2, NULL);

    pthread_join(t2, NULL);

}

output:

thread1
thread1 hey hey
thread2

common deadlock example, no problem.

2. recursive mutex with deadlock

Just uncomment this line
error = pthread_mutexattr_settype(&lock_attr, PTHREAD_MUTEX_RECURSIVE);
and comment out the other one.

output:

thread1
thread1 hey hey
thread2

Yes, recursive mutex can also cause deadlock.

3. normal mutex, relock in the same thread

#include <pthread.h>
#include <stdio.h>
#include <unistd.h>

pthread_mutex_t lock;


void func3(){
    printf("func3\n");
    pthread_mutex_lock(&lock);
    printf("func3 hey hey\n");
}

void * func1(void *arg){
    printf("thread1\n");
    pthread_mutex_lock(&lock);
    func3();
    printf("thread1 hey hey\n");

}


void * func2(void *arg){
    printf("thread2\n");
    pthread_mutex_lock(&lock);
    printf("thread2 hey hey\n");
}

int main(){
    pthread_mutexattr_t lock_attr;
    int error;
//    error = pthread_mutexattr_settype(&lock_attr, PTHREAD_MUTEX_RECURSIVE);
    error = pthread_mutexattr_settype(&lock_attr, PTHREAD_MUTEX_DEFAULT);
    if(error){
        perror(NULL);
    }

    pthread_mutex_init(&lock, &lock_attr);

    pthread_t t1, t2;

    pthread_create(&t1, NULL, func1, NULL);
    sleep(2); 
    pthread_create(&t2, NULL, func2, NULL);

    pthread_join(t2, NULL);

}

output:

thread1
func3
thread2

Deadlock in thread t1, in func3.
(I use sleep(2) to make it easier to see that the deadlock is firstly caused by relocking in func3)

4. recursive mutex, relock in the same thread

Again, uncomment the recursive mutex line and comment out the other line.

output:

thread1
func3
func3 hey hey
thread1 hey hey
thread2

Deadlock in thread t2, in func2. See? func3 finishes and exits, relocking does not block the thread or lead to deadlock.

---

So, last question, why do we need it ?

For recursive function (called in multi-threaded programs and you want to protect some resource/data).

E.g. You have a multi thread program, and call a recursive function in thread A. You have some data that you want to protect in that recursive function, so you use the mutex mechanism. The execution of that function is sequential in thread A, so you would definitely relock the mutex in recursion. Use normal mutex causes deadlocks. And resursive mutex is invented to solve this.

See an example from the accepted answer When to use recursive mutex?.

The Wikipedia explains the recursive mutex very well. Definitely worth for a read. Wikipedia: Reentrant_mutex

Officialdom