Read preferring RW mutex lock in Golang

Question

I need a read preferring RW mutex in golang. Is there a package in golang that will satisfy my needs. I tried sync.RWMutex, but it seems to be write preferring lock. Here goes my attempt to distinguish Go's RWMutex,

package main

import (
    "fmt"
    "sync"
    "time"
)

func main() {

y := &resource{x: 10}

go func() {
    defer fmt.Println("done first read")
    y.RLock()
    defer y.RUnlock()
    go func() {
        defer fmt.Println("done first write")
        fmt.Println("first write req")
        y.Lock()
        fmt.Println("after first write granted")
        defer y.Unlock()
    }()
    time.Sleep(time.Second)
    go func() {
        defer fmt.Println("done second read")
        fmt.Println("second read req")
        y.RLock()
        fmt.Println("after second read granted")
        defer y.RUnlock()
    }()

    time.Sleep(10 * time.Second)
}()

time.Sleep(time.Minute)

}

type resource struct {
    sync.RWMutex
    x int
}

Output:

first write req
second read req
done first read
after first write granted
done first write
after second read granted
done second read

Second reader is kept waiting till the writer releases the lock.

Cabinetwork

Answer 1

sync.RWMutex implements both write preferred and read preferred locking. It all depends on how you use it to gain either write preferred or read preferred.

Taking your wikipedia link pseudo code as an example of Lock-For-Read (in a read-preferred situation):

* Input: mutex m, condition variable c, integer r (number of readers waiting), flag w (writer waiting).
* Lock m (blocking).
* While w:
* wait c, m[a]
* Increment r.
* Unlock m.

And the Lock-For-Write pattern in a read-preferred sistuation as long as you are following the pattern above for Lock-For-Reads:

* Lock m (blocking).
* While (w or r > 0):
* wait c, m
* Set w to true.
* Unlock m.

You can see this mechanism at play within how the RWMutex is implemented. Remember, the Go framework is just Go code - view the code to see how it is implemented:

https://golang.org/src/sync/rwmutex.go?s=879:905#L20

29  // RLock locks rw for reading.
30  func (rw *RWMutex) RLock() {
31      if race.Enabled {
32          _ = rw.w.state
33          race.Disable()
34      }
35      if atomic.AddInt32(&rw.readerCount, 1) < 0 {
36          // A writer is pending, wait for it.
37          runtime_Semacquire(&rw.readerSem)
38      }
39      if race.Enabled {
40          race.Enable()
41          race.Acquire(unsafe.Pointer(&rw.readerSem))
42      }
43  }

A key to note is the rw.readerSem in the code above which gives you your integer r in the wikipedia example pattern, which languages (like Go and others) call a Semaphore:

http://www.golangpatterns.info/concurrency/semaphores

The real meat of of the wait is on line 37, for the runtime_Semaquire():

https://golang.org/src/sync/runtime.go

11  // Semacquire waits until *s > 0 and then atomically decrements it.
12  // It is intended as a simple sleep primitive for use by the synchronization
13  // library and should not be used directly.
14  func runtime_Semacquire(s *uint32)

Knowing that, and seeing how a RWMutex.RLock() increments a read that number, you can refactor your code accordingly.

Take a look at how RWMutex.RUnlock decrements that but most importantly how RWMutex.Lock() forces the wait for all active readers:

71  // Lock locks rw for writing.
72  // If the lock is already locked for reading or writing,
73  // Lock blocks until the lock is available.
74  // To ensure that the lock eventually becomes available,
75  // a blocked Lock call excludes new readers from acquiring
76  // the lock.
77  func (rw *RWMutex) Lock() {
78      if race.Enabled {
79          _ = rw.w.state
80          race.Disable()
81      }
82      // First, resolve competition with other writers.
83      rw.w.Lock()
84      // Announce to readers there is a pending writer.
85      r := atomic.AddInt32(&rw.readerCount, -rwmutexMaxReaders) + rwmutexMaxReaders
86      // Wait for active readers.
87      if r != 0 && atomic.AddInt32(&rw.readerWait, r) != 0 {
88          runtime_Semacquire(&rw.writerSem)
89      }
90      if race.Enabled {
91          race.Enable()
92          race.Acquire(unsafe.Pointer(&rw.readerSem))
93          race.Acquire(unsafe.Pointer(&rw.writerSem))
94      }
95  }

That's most likely why you are seeing the 2nd reader waiting.

Remember, the Semaphores are shared across not just the RWMutex instance you created, but also throughout the runtime to schedule around other goroutines and other locks. Hence why trying to force a pattern can do more harm than good overall in an app.

My advise would be to take a step back and consider why you want read-preferred locking in your architecture at all. Are you truly at a performance level that CPU context switching is slowing down your high frequency application? I'd say there's a more systematic approach that could be taken instead of trying to implement a 'read-preferred locking` pattern just because it sounds cool and sounds like it solve all your problems. What are your benchmarks numbers? What is the size of input data, and across how many concurrent processes? Does it have to be shared? Is it under X GB of memory consumption and can you switch to putting things on the stack (e.g. channels, no mutex locking)? What about the read data on the stack and keep a write set for locking? For how long until the GC cleans up the stack vs having to keep things on the heap? Etc etc.

Nikolai

Answer 2

Seems you can achieve desired behaviour with sync.WaitGroup sync primitive for example

var wg sync.WaitGroup
go func() {
            defer fmt.Println("done second read")
            fmt.Println("second read req")
            y.RLock()   //wait writer
            wg.Add(1)   //report busy
            fmt.Println("after second read granted")
            defer wg.Done() //report done
            defer y.RUnlock()
        }()
//forcing writer to wait all readers
go func() {
            defer fmt.Println("done first write")
            fmt.Println("first write req")
            wg.Wait()  //wait all readers
            y.Lock()
            fmt.Println("after first write granted")
            defer y.Unlock()
        }()

You can try https://play.golang.org/p/y831xIrglj

Fabrice

Nice. I don't use mutex's a lot and just gave the long spill on how one would research it. – Nikolai