When to use Semaphore instead of Dispatch Group?

Question

I would assume that I am aware of how to work with DispatchGroup, for understanding the issue, I've tried:

class ViewController: UIViewController {
    override func viewDidLoad() {
        super.viewDidLoad()

        performUsingGroup()
    }

    func performUsingGroup() {
        let dq1 = DispatchQueue.global(qos: .userInitiated)
        let dq2 = DispatchQueue.global(qos: .userInitiated)

        let group = DispatchGroup()

        group.enter()
        dq1.async {
            for i in 1...3 {
                print("\(#function) DispatchQueue 1: \(i)")
            }
            group.leave()
        }

        group.wait()

        dq2.async {
            for i in 1...3 {
                print("\(#function) DispatchQueue 2: \(i)")
            }
        }

        group.notify(queue: DispatchQueue.main) {
            print("done by group")
        }
    }
}

and the result -as expected- is:

performUsingGroup() DispatchQueue 1: 1
performUsingGroup() DispatchQueue 1: 2
performUsingGroup() DispatchQueue 1: 3
performUsingGroup() DispatchQueue 2: 1
performUsingGroup() DispatchQueue 2: 2
performUsingGroup() DispatchQueue 2: 3
done by group

For using the Semaphore, I implemented:

func performUsingSemaphore() {
    let dq1 = DispatchQueue.global(qos: .userInitiated)
    let dq2 = DispatchQueue.global(qos: .userInitiated)

    let semaphore = DispatchSemaphore(value: 1)

    dq1.async {
        semaphore.wait()
        for i in 1...3 {
            print("\(#function) DispatchQueue 1: \(i)")
        }
        semaphore.signal()
    }

    dq2.async {
        semaphore.wait()
        for i in 1...3 {
            print("\(#function) DispatchQueue 2: \(i)")
        }
        semaphore.signal()
    }
}

and called it in the viewDidLoad method. The result is:

performUsingSemaphore() DispatchQueue 1: 1
performUsingSemaphore() DispatchQueue 1: 2
performUsingSemaphore() DispatchQueue 1: 3
performUsingSemaphore() DispatchQueue 2: 1
performUsingSemaphore() DispatchQueue 2: 2
performUsingSemaphore() DispatchQueue 2: 3

Conceptually, both of DispachGroup and Semaphore serve the same purpose (unless I misunderstand something).

Honestly, I am unfamiliar with: when to use the Semaphore, especially when workin with DispachGroup -probably- handles the issue.

What is the part that I am missing?

Abutilon

Semaphores can be used for diverse array of applications. But for tasks like this, anything that calls wait (whether semaphore or group) is generally a bad idea. You're blocking a thread when you do that (and, worse, if you block the main thread, that can cause serious problems). It's almost always better to use dispatch group notify, or similar patterns. Embrace asynchronous patterns rather than using wait to achieve synchronous behaviors. – Ked

I imagine that in the semaphore case there's theoretically a chance that dq2 runs before dq1, in which case the order of execution isn't guaranteed (unlike with DispatchGroup). Can anyone confirm/correct? – Alkoran

@GuyKogus - The potential timing of dq1 vs. dq2 doesn't have anything to do with the semaphore. It's the fact that (a) there are two separate queues and (b) he's doing async. – Ked

I just did a quick test in Playgrounds and DispatchQueue.global(qos: .userInitiated) will return the same queue. So in that case, if that queue is serial, the semaphores won't be doing anything. Unless that code's just an example and we should imagine that they're unique queues. – Alkoran

@GuyKogus "if that queue is serial" ... Global queues are not serial. – Ked

dq1and dq2are the same queue in the code above. – Missymist

Answer 1

Conceptually, both of DispatchGroup and Semaphore serve the same purpose (unless I misunderstand something).

The above is not exactly true. You can use a semaphore to do the same thing as a dispatch group but it is much more general.

Dispatch groups are used when you have a load of things you want to do that can all happen at once, but you need to wait for them all to finish before doing something else.

Semaphores can be used for the above but they are general purpose synchronisation objects and can be used for many other purposes too. The concept of a semaphore is not limited to Apple and can be found in many operating systems.

In general, a semaphore has a value which is a non negative integer and two operations:

• wait If the value is not zero, decrement it, otherwise block until something signals the semaphore.

• signal If there are threads waiting, unblock one of them, otherwise increment the value.

Needless to say both operations have to be thread safe. In olden days, when you only had one CPU, you'd simply disable interrupts whilst manipulating the value and the queue of waiting threads. Nowadays, it is more complicated because of multiple CPU cores and on chip caches etc.

A semaphore can be used in any case where you have a resource that can be accessed by at most N threads at the same time. You set the semaphore's initial value to N and then the first N threads that wait on it are not blocked but the next thread has to wait until one of the first N threads has signaled the semaphore. The simplest case is N = 1. In that case, the semaphore behaves like a mutex lock.

A semaphore can be used to emulate a dispatch group. You start the sempahore at 0, start all the tasks - tracking how many you have started and wait on the semaphore that number of times. Each task must signal the semaphore when it completes.

However, there are some gotchas. For example, you need a separate count to know how many times to wait. If you want to be able to add more tasks to the group after you have started waiting, the count can only be updated in a mutex protected block and that may lead to problems with deadlocking. Also, I think the Dispatch implementation of semaphores might be vulnerable to priority inversion. Priority inversion occurs when a high priority thread waits for a resource that a low priority has grabbed. The high priority thread is blocked until the low priority thread releases the resource. If there is a medium priority thread running, this may never happen.

You can pretty much do anything with a semaphore that other higher level synchronisation abstractions can do, but doing it right is often a tricky business to get right. The higher level abstractions are (hopefully) carefully written and you should use them in preference to a "roll your own" implementation with semaphores, if possible.

Nodular

Answer 2

Semaphores and groups have, in a sense, opposite semantics. Both maintain a count. With a semaphore, a wait is allowed to proceed when the count is non-zero. With a group, a wait is allowed to proceed when the count is zero.

A semaphore is useful when you want to set a maximum on the number of threads operating on some shared resource at a time. One common use is when the maximum is 1 because the shared resource requires exclusive access.

A group is useful when you need to know when a bunch of tasks have all been completed.

Dogger

This is the most succinct explanation. I wish it was the top voted answer! – Invalidity

One? In all the examples I see where DispatchSemaphore emulates a DispatchGroup, the initial value is 0. Could you please explain this? Thanks a lot! – Slavey

At no point am I talking about emulating a group with a semaphore. And that sentence begins with "One common use…"; it doesn't exclude other possible uses. – Dogger

Answer 3

Use a semaphore to limit the amount of concurrent work at a given time. Use a group to wait for any number of concurrent work to finish execution.

In case you wanted to submit three jobs per queue it should be

import Foundation

func performUsingGroup() {
    let dq1 = DispatchQueue(label: "q1", attributes: .concurrent)
    let dq2 = DispatchQueue(label: "q2", attributes: .concurrent)
    let group = DispatchGroup()
    
    for i in 1...3 {
        group.enter()
        dq1.async {
            print("\(#function) DispatchQueue 1: \(i)")
            group.leave()
        }
    }
    for i in 1...3 {
        group.enter()
        dq2.async {
            print("\(#function) DispatchQueue 2: \(i)")
            group.leave()
        }
    }
    
    group.notify(queue: DispatchQueue.main) {
        print("done by group")
    }
}

performUsingGroup()
RunLoop.current.run(mode: RunLoop.Mode.default,  before: Date(timeIntervalSinceNow: 1))

and

import Foundation

func performUsingSemaphore() {
    let dq1 = DispatchQueue(label: "q1", attributes: .concurrent)
    let dq2 = DispatchQueue(label: "q2", attributes: .concurrent)
    let semaphore = DispatchSemaphore(value: 1)
    
    for i in 1...3 {
        dq1.async {
            _ = semaphore.wait(timeout: DispatchTime.distantFuture)
            print("\(#function) DispatchQueue 1: \(i)")
            semaphore.signal()
        }
    }
    for i in 1...3 {
        dq2.async {
            _ = semaphore.wait(timeout: DispatchTime.distantFuture)
            print("\(#function) DispatchQueue 2: \(i)")
            semaphore.signal()
        }
    }
}

performUsingSemaphore()
RunLoop.current.run(mode: RunLoop.Mode.default,  before: Date(timeIntervalSinceNow: 1))

Alike

This is better explanation that mine :) – Expand

Two thoughts: 1. FYI, if the dispatched tasks are, themselves, synchronous, you can replace group.enter() and group.leave() with dq1.async(group: group). 2. The semaphore pattern is dangerous because if there were hundreds of tasks rather than just 3, you're going block all of the worker threads. It's better to pull the wait calls out of those async calls, but have a single task that does the wait calls, that way, you're blocking, at max, only one thread and cannot exhaust the worker threads. See gist.github.com/robertmryan/deaf98b6095a1545ebdab44d0ef3d5af. – Ked

dq1 and dq2 are actually the same queue instance, the global queue for default qos. – Missymist

Answer 4

The replies above by Jano and Ken are correct regarding 1) the use of semaphore to limit the amount of work happening at once 2) the use of a dispatch group so that the group will be notified when all the tasks in the group are done. For example, you may want to download a lot of images in parallel but since you know that they are heavy images, you want to limit to two downloads only at a single time so you use a semaphore. You also want to be notified when all the downloads (say there are 50 of them) are done, so you use DispatchGroup. Thus, it is not a matter of choosing between the two. You may use one or both in the same implementation depending on your goals. This type of example was provided in the Concurrency tutorial on Ray Wenderlich's site:

let group = DispatchGroup()
let queue = DispatchQueue.global(qos: .utility)
let semaphore = DispatchSemaphore(value: 2)

let base = "https://yourbaseurl.com/image-id-"
let ids = [0001, 0002, 0003, 0004, 0005, 0006, 0007, 0008, 0009, 0010, 0011, 0012]

var images: [UIImage] = []

for id in ids {
  guard let url = URL(string: "\(base)\(id)-jpeg.jpg") else { continue }
  
  semaphore.wait()
  group.enter()
  
  let task = URLSession.shared.dataTask(with: url) { data, _, error in
    defer {
      group.leave()
      semaphore.signal()
    }
    
    if error == nil,
      let data = data,
      let image = UIImage(data: data) {
      images.append(image)
    }
  }
  
  task.resume()
}

Zymosis

I was able to resolve my situation(https://mcmap.net/q/340366/-dropbox-files-download-does-not-start-when-number-of-files-in-folder-is-gt-1000/14414215) by using semaphore and your answer. – Arvind

Answer 5

One typical semaphore use case is a function that can be called simultaneously from different threads and uses a resource that should not be called from multiple threads at the same time:

func myFunction() {
    semaphore.wait()
    // access the shared resource
    semaphore.signal()
}

In this case you will be able to call myFunction from different threads but they won't be able to reach the locked resource simultaneously. One of them will have to wait until the second one finishes its work.

A semaphore keeps a count, therefore you can actually allow for a given number of threads to enter your function at the same time.

Typical shared resource is the output to a file.

A semaphore is not the only way to solve such problems. You can also add the code to a serial queue, for example.

Semaphores are low level primitives and most likely they are used a lot under the hood in GCD.

Another typical example is the producer-consumer problem, where the signal and wait calls are actually part of two different functions. One which produces data and one which consumes them.

Expand

Queues will work so long as they are serial, otherwise there's a potential for race conditions. – Alkoran

@GuyKogus I should have been more precise. Most people deal with such things by adding them to the main queue which is serial and that would do the same as semaphore with count 1. That wouldn't work efficiently for a multiple producer problem for example since the work is essentially moved to one thread. – Expand

Answer 6

Generally semaphore can be considered mainly that we can solve the critical section problem. Locking certain resource to achieve synchronisation. Also what happens if sleep() is invoked, can we achieve the same thing by using a semaphore ?

Dispatch groups we will use when we have multiple group of operations to be carried out and we need a tracking or set dependencies each other or notification when a group os tasks finishes its execution.

Rhapsodize