Why this example using mutex is less efficient compared to another one with additional condition variable?

Question

An example from The Linux Programming Interface:

In the producer threads, we would have code such as the following:

static pthread_mutex_t mtx = PTHREAD_MUTEX_INITIALIZER;
static int avail = 0;

/* Code to produce a unit omitted */
s = pthread_mutex_lock(&mtx);
if (s != 0)
    errExitEN(s, "pthread_mutex_lock");

avail++; /* Let consumer know another unit is available */

s = pthread_mutex_unlock(&mtx);
if (s != 0)
    errExitEN(s, "pthread_mutex_unlock");

And in the main (consumer) thread, we could employ the following code:

for (;;) {
    s = pthread_mutex_lock(&mtx);
    if (s != 0)
        errExitEN(s, "pthread_mutex_lock");
    while (avail > 0) {
        /* Consume all available units */
        /* Do something with produced unit */
        avail--;
    }
    s = pthread_mutex_unlock(&mtx);
    if (s != 0)
        errExitEN(s, "pthread_mutex_unlock");
}

The above code works, but it wastes CPU time, because the main thread continually loops, checking the state of the variable avail. A condition variable remedies this problem. It allows a thread to sleep (wait) until another thread notifies (signals) it that it must do something.

---

The book then gives what he thinks is a better version, with condition variable:

static pthread_mutex_t mtx = PTHREAD_MUTEX_INITIALIZER;
static pthread_cond_t cond = PTHREAD_COND_INITIALIZER;
static int avail = 0;

Producer thread:

/* Code to produce a unit omitted */
s = pthread_mutex_lock(&mtx);
if (s != 0)
    errExitEN(s, "pthread_mutex_lock");

avail++;  /* Let consumer know another unit is available */

s = pthread_mutex_unlock(&mtx);
if (s != 0)
    errExitEN(s, "pthread_mutex_unlock");

s = pthread_cond_signal(&cond); /* Wake sleeping consumer */
if (s != 0)
    errExitEN(s, "pthread_cond_signal");

Consumer thread:

for (;;) {
    s = pthread_mutex_lock(&mtx);
    if (s != 0)
        errExitEN(s, "pthread_mutex_lock");
    while (avail == 0) {            /* Wait for something to consume */
        s = pthread_cond_wait(&cond, &mtx);
        if (s != 0)
            errExitEN(s, "pthread_cond_wait");
    }

    while (avail > 0) {             /* Consume all available units */
        /* Do something with produced unit */
        avail--;
    }

    s = pthread_mutex_unlock(&mtx);
    if (s != 0)
        errExitEN(s, "pthread_mutex_unlock");
    /* Perhaps do other work here that doesn't require mutex lock */
}

---

Question: Why the first version is less effcient(wastes CPU time) compared to the second one? I can't see the difference.

Btw, could you give me an example which may illustrate the author's point of view, which I think is:
You can use mutex along with condition variable to improve perfomance, compared to the one that uses only mutex.

Swashbuckler

To me, it seems that everything is already clearly explained in the question. Obviously, assuming there is no data to consume, first code sample will loop forever waiting data to be available while in the second case the thread will be suspended while waiting the condition variable. A thread that sleep waiting clearly wastes less CPU than one that do busy waiting testing a mutex at a fast rate. In first case, the mutex only wait producer mutex is unlock not that data is available. – Socage

@Socage How will the first code sample loop forever since it's blocked by trying to lock a mutex owned by the producer? There is no busy test in the loop of the first version. The consumer is blocked -> producer increase avail, unblocks the mutex -> consumer continues. – Swashbuckler

You assume that the producer always have the mutex locked. When it is not the case, the consumer might be able to loop hundreds of time when there is no available data (let say that the producer take a lot of time to produce an item and then lock the mutex a short time to insert the item). – Socage

@Socage Yes you are right :(. Now I get it. Thanks. I suggest making it as an answer and then I can accept and close this question. – Swashbuckler

Answer 1

To start off, both are problematic. You want the producer to produce items outside the lock, and use the lock only to enqueue the job and notify the consumer about it, and you weant the consumer to wait and dequeue inside the lock, but to "do something" with the job OUTSIDE of the lock.

Now, regarding the condition variable: A mutex allows you to synchronize the code of two or more threads to ensure that the code within the mutex runs with no interruptions in order to maintain data integrity: if 2 threads run i++ at the same time you have no guarantee that i will grow exactly by 2. So the mutex here solves the problem of enqueuing/dequeuing from one queue (or in this case the avail variable) at the same time, but you do not have a mechanism to actually let the consumer sleep so not to waste precious CPU that can be used by the producer in order to produce more work. This is where the condition variable comes to help the consumer to sleep until the producers sets avail>0 and wakes the consumer up.

Example producer:

while (true) {
    // produce outside the lock
    ...

    pthread_mutex_lock(&mtx);   // lock
    ++avail;                    // update new job 
    pthread_cond_signal(&cond); // wake up consumer
    pthread_mutex_unlock(&mtx); // unlock
}

Example consumer:

while (true) {
    pthread_mutex_lock(&mtx);   // lock
    while (avail<=0)            // wait for new job
        pthread_cond_wait(&cond, &mtx);
    --avail;                    // get the new job 
    pthread_mutex_unlock(&mtx); // unlock

    // consume job outside the lock
}

Note that I have removed error handling for simplification

For more reading

Steamboat

Thank you first. "but you do not have a mechanism to actually let the consumer sleep so not to waste precious CPU that can be used by the producer in order to produce more work.". Doesn't the consumer thread in the first version sleep when it's blocked by trying to lock a mutex owned by the producer? – Swashbuckler

From your answer, you seem to indicate that a thread blocked by trying to lock a locked mutex and pthread_condi_wait have different low-level effects. Could you explain a bit more about this? – Swashbuckler

Your mutex is unlocked and them immediately locked again to continue producing, so the consumer doesn't have time to lock it. – Steamboat

Think of a mutex as a resource lock. You want to make sure that only 1 thread is using the resource. Your producer is never done using the resource, it wants to continue producing. A condition variable has no ties to a resource, it is a means for 1 thread to signal to another thread to wake up. There is no resource ownership here. – Steamboat