Calling pthread_cond_signal without locking mutex

Question

I read somewhere that we should lock the mutex before calling pthread_cond_signal and unlock the mutex after calling it:

The pthread_cond_signal() routine is used to signal (or wake up) another thread which is waiting on the condition variable. It should be called after mutex is locked, and must unlock mutex in order for pthread_cond_wait() routine to complete.

My question is: isn't it OK to call pthread_cond_signal or pthread_cond_broadcast methods without locking the mutex?

Leatherback

Answer 1

If you do not lock the mutex in the codepath that changes the condition and signals, you can lose wakeups. Consider this pair of processes:

Process A:

pthread_mutex_lock(&mutex);
while (condition == FALSE)
    pthread_cond_wait(&cond, &mutex);
pthread_mutex_unlock(&mutex);

Process B (incorrect):

condition = TRUE;
pthread_cond_signal(&cond);

Then consider this possible interleaving of instructions, where condition starts out as FALSE:

Process A                             Process B

pthread_mutex_lock(&mutex);
while (condition == FALSE)

                                      condition = TRUE;
                                      pthread_cond_signal(&cond);

pthread_cond_wait(&cond, &mutex);

The condition is now TRUE, but Process A is stuck waiting on the condition variable - it missed the wakeup signal. If we alter Process B to lock the mutex:

Process B (correct):

pthread_mutex_lock(&mutex);
condition = TRUE;
pthread_cond_signal(&cond);
pthread_mutex_unlock(&mutex);

...then the above cannot occur; the wakeup will never be missed.

(Note that you can actually move the pthread_cond_signal() itself after the pthread_mutex_unlock(), but this can result in less optimal scheduling of threads, and you've necessarily locked the mutex already in this code path due to changing the condition itself).

Springtail

@Nemo: Yes, in the "correct" path the pthread_signal_cond() can be moved after the mutex unlock, although it is probably better not to. It is perhaps more correct to say that at the point where you are calling pthread_signal_cond(), you will have already needed to have locked the mutex to modify the condition itself. – Springtail

@Nemo: Yes, the code is broken, which is why it's marked as "incorrect" - it's my experience that people who are asking this question are often considering leaving the locking out entirely on the signalling path. Perhaps your experience differs. – Springtail

I misread what you were saying. I thought you were arguing that signaling the condition after unlocking the mutex was incorrect because of interleaving... I realize I was mis-reading your answer, so I deleted my other comments. Apologies. – Interlocutress

@Nemo: I think you're right that my answer was at least unclear, so I've updated it to hopefully improve it. – Springtail

For what it's worth, it seems the question of whether to unlock before or after the signal is actually non-trivial to answer. Unlocking after ensures that a lower-priority thread won't be able to steal the event from a higher-priority one, but if you're not using priorities, unlocking before the signal will actually reduce the number of system calls/context switches and improve overall performance. – Tartu

I want to ask about this alternative: codecondition = TRUE;pthread_mutex_lock(&mutex);pthread_cond_signal(&cond);pthread_mutex_unlock(&mutex); code Is there any situation where this would be problematic? – Dissolve

At least, I does not find any scenario where a deadlock can occur. – Dissolve

@dada That's just undefined behavior. POSIX prohibits modifying one object while another thread is, or might be, accessing it. Since you hold no lock and condition is shared, you violate this rule since you modify it while another thread might be accessing it. The issue isn't deadlock specifically but that anything can happen. – Pudens

@R.. You have it backwards. Unlocking before the signal increases the number of system calls and decreases overall performance. If you signal before the unlock, the implementation knows the signal can't possibly wake a thread (because any thread blocked on the cv needs the mutex to make forward progress), allowing it to use the fast path. If you signal after the unlock, both the unlock and the signal can wake threads, meaning they're both expensive operations. – Pudens

@DavidSchwartz: I think both those claims are incorrect, but it's a complex topic I'd rather not write up only to have it buried in a long thread of comments that might get lost... If you want to discuss it, maybe open a new question about which is more efficient? – Tartu

@R.. I know the answer. I've studied it extensively. If you signal before you unlock, the signal is almost a no-op, virtually free (since it can't make any thread ready-to-run and you already hold the lock protecting most of the state you need to change). If you signal after the unlock, both the signal and the unlock are complex and can potentially unblock other threads. – Pudens

@DavidSchwartz: R. is the author of a widely-used pthreads implementation, so it seems probable that he also has some insight into this issue that shouldn't be so lightly dismissed. – Springtail

@Springtail That's pretty strange then. Because it's not particularly difficult to see that if you signal while holding the mutex, that signal can't possibly change any thread's state, making it much less expensive. There's no cost that balances that out anywhere else in the equation. (Any thread blocked on the condition variable can't make forward progress while the mutex is locked anyway because it must re-acquire the mutex.) (Which pthread library? I'll benchmark it.) – Pudens

@DavidSchwartz: If the mutex is a normal type, which usually doesn't track its owner because doing so is slightly more costly, there is no way for pthread_cond_signal to determine if the mutex is locked by the calling thread or another thread, so it can't make any assumption. – Tartu

@R.. You can just use an illegal thread ID to represent "unlocked" and atomically compare/replace with your thread ID to lock a mutex. It's no more expensive than any other method, except you do have to look up your own thread ID. But that's nearly free on most platforms. But in any event, it's quite silly to base your design on what one implementation happens to be able to optimize. Algorithmically, it's more efficient because it can't wake a thread. And you should code based on efficient algorithms, not implementation details. – Pudens

@DavidSchwartz: As I said this is a complicated topic and it's really a shame you insisted on burying it in a comment thread on the assumption that you're right. It's possible to write a normal mutex that tracks owner but it's a lot more expensive than you think, due to requirements on what happens when a thread exits while owning a mutex. It also doesn't seem to help, because you can't actually defer the work of pthread_cond_signal to mutex unlock time. It's valid to destroy the condvar as soon as pthread_cond_signal returns if there are provably no remaining waiters. – Tartu

Also, even if you could defer the work of the signal to mutex unlock time, a mutex can be used simultaneously with unboundedly many condvars, so mutex unlock would become O(n) rather than O(1) which is highly undesirable from a standpoint of realtime properties. – Tartu

@R.. Yeah, you were right, this isn't the place for it. But I can't imagine any mainstream implementation would optimize for relatime properties for mutexes with lots of condvars associated with them over optimizing signaling a condvar. – Pudens

@DavidSchwartz: The last thing about O(n) mutex performance was the least of the problems. More important is that the implementation you want just isn't possible because of lifetime considerations. The whole thing is an interesting topic and I wish you'd taken me up on opening a proper question on it. – Tartu

I might be a bit late, but I was wondering since no one actually touched that point. If there truly is optimization from implementations when signaling on conditions and unlocking mutexes, I'm really interested in how the system knows which cond_wait was called with which mutex (since, in contrast with what R stated, you can have a cond var used with many mutexes.. or so I think). Does it simply store a cond/mutex combination for each cond_wait call? -- Thx – Gargantua

@Alceste_: See this text in POSIX: "When a thread waits on a condition variable, having specified a particular mutex to either the pthread_cond_timedwait() or the pthread_cond_wait() operation, a dynamic binding is formed between that mutex and condition variable that remains in effect as long as at least one thread is blocked on the condition variable. During this time, the effect of an attempt by any thread to wait on that condition variable using a different mutex is undefined." – Springtail

The mutex is required in process B to ensure that the modification to condition is visible to process A. Technically, the memory fence instruction executed in the implementation of the mutex unlock function ensures that the modification to condition happens before the signaling of the condition variable; the memory fence instruction executed by the mutex lock function in A ensures that it cannot see the signaled condition variable without also seeing the change to condition. – Twostep

If both process A & B start with pthread_mutex_lock(&mutex); will not Process B get blocked. In Process A while loop will be executing and in Process B when pthread_mutex_lock(&mutex); it will be blocked. So above example will be a deadlock. Please correct me if I am wrong. – Colligate

@beemavishnu: No, there is no deadlock, because pthread_cond_wait() atomically releases the mutex and waits on the condition variable. It re-aquires the mutex after being woken and before returning. That's why the locked mutex has to be passed to that function. – Springtail

@caf: Thank you for your reply. By default assume variable 'condition' is false. In Process A, pthread_cond_wait(&cond, &mutex) will not be called because of check in while loop "while (condition == FALSE)". So pthread_cond_wait() will not release mutex. Whereas in Process B (correct), "condition = true" is inside pthread_mutex_lock(&mutex). Because of this Process A and Process B (correct) will be in dead lock as both will call pthread_mutex_lock(&mutex). To avoid this, "condition = true" should be moved out of pthread_mutex_lock(&mutex) in Process B (correct). – Colligate

@beemavishnu: When condition is false, the test in while (condition == FALSE) is true, and so the while() body is executed, which means the pthread_cond_wait() is called and the mutex is released while the thread waits. The pthread_cond_wait() call is in the body of the while() loop (it is the only statement in the body of the loop). – Springtail

@caf: Sorry, I overlooked while loop. Thanks a lot for clarifying. – Colligate

Answer 2

According to this manual :

The pthread_cond_broadcast() or pthread_cond_signal() functions may be called by a thread whether or not it currently owns the mutex that threads calling pthread_cond_wait() or pthread_cond_timedwait() have associated with the condition variable during their waits; however, if predictable scheduling behavior is required, then that mutex shall be locked by the thread calling pthread_cond_broadcast() or pthread_cond_signal().

The meaning of the predictable scheduling behavior statement was explained by Dave Butenhof (author of Programming with POSIX Threads) on comp.programming.threads and is available here.

Whallon

+1 for linking the mail from Dave Butenhof. I always wondered about this issue myself, now I know... Learned something important today. Thanks. – Spurgeon

If predictable scheduling behavior is required, then put the statements in one thread in the order that you want, or use thread priorities. – Ambroid

Answer 3

caf, in your sample code, Process B modifies condition without locking the mutex first. If Process B simply locked the mutex during that modification, and then still unlocked the mutex before calling pthread_cond_signal, there would be no problem --- am I right about that?

I believe intuitively that caf's position is correct: calling pthread_cond_signal without owning the mutex lock is a Bad Idea. But caf's example is not actually evidence in support of this position; it's simply evidence in support of the much weaker (practically self-evident) position that it is a Bad Idea to modify shared state protected by a mutex unless you have locked that mutex first.

Can anyone provide some sample code in which calling pthread_cond_signal followed by pthread_mutex_unlock yields correct behavior, but calling pthread_mutex_unlock followed by pthread_cond_signal yields incorrect behavior?

Atomism

Actually, I think my question is a duplicate of this one, and the answer is "It's fine, you can totally call pthread_cond_signal without owning the mutex lock. There's no correctness issue. But in common implementations you'll miss out on a clever optimization deep inside pthreads, so it's slightly preferable to call pthread_cond_signal while still holding the lock." – Atomism

I made this observation in the last paragraph of my answer. – Springtail

You have a good scenario here: https://mcmap.net/q/218956/-signal-and-unlock-order Note that it does not claim the behaviour is incorrect, it only presents a case where the behaviour might not be as expected. – Recrement

It is possible to create sample code where calling pthread_cond_signal after pthread_mutex_unlock can result in a lost wakeup because the signal is caught be the "wrong" thread, one that blocked after seeing the change in the predicate. This is only an issue if the same condition variable can be used for more than one predicate and you don't use pthread_cond_broadcast, which is a rare and fragile pattern anyway. – Pudens