Wait for all Operations in queue to finish before performing task

Question

I have an Operation subclass and Operation queue with maxConcurrentOperationCount = 1.

This performs my operations in a sequential order that i add them which is good but now i need to wait until all operations have finished before running another process.

i was trying to use notification group but as this is run in a for loop as soon as the operations have been added to the queue the notification group fires.. How do i wait for all operations to leave the queue before running another process?

for (index, _) in  self.packArray.enumerated() {

    myGroup.enter()
    let myArrayOperation = ArrayOperation(collection: self.outerCollectionView, id: self.packArray[index].id, count: index)
    myArrayOperation.name = self.packArray[index].id
    downloadQueue.addOperation(myArrayOperation)
    myGroup.leave()

}

myGroup.notify(queue: .main) {
 // do stuff here
}

Overbuild

Put the myGroup.leave() in the completion block for the operation. – Kabob

Use waitUntilAllOperationsAreFinished developer.apple.com/reference/foundation/operationqueue/… – Saleme

Please note: your queue is guaranteed to execute operations in submission order only if all operations in the queue have the same relative queuePriority and become ready in the order they were added to the serial queue. See the "Determine the Execution Order" section in the OperationQueue docs here – Marris

Answer 1

You can use operation dependencies to initiate some operation upon the completion of a series of other operations:

let queue = OperationQueue()

let completionOperation = BlockOperation {
    // all done
}

for object in objects {
    let operation = ...
    completionOperation.addDependency(operation)
    queue.addOperation(operation)
}

OperationQueue.main.addOperation(completionOperation)  // or, if you don't need it on main queue, just `queue.addOperation(completionOperation)`

Or, in iOS 13 and later, you can use barriers:

let queue = OperationQueue()

for object in objects {
    queue.addOperation(...)
}

queue.addBarrierBlock {
    DispatchQueue.main.async {
        // all done
    }
}

Shabbir

Very clean. Thanks – Overbuild

Hey Rob, I've done exactly this but my completionOperation is still being prematurely fired. My operations are asynchronous network operations so an operation is only considered finished when the response comes back. I read that I need to override the isAsynchronous property among a few others to make it an asynchronous operation, but I read that the property will be ignored if I add the operation to a queue anyway. So I'm confused on what to do. Could you please advise further? – Earflap

what if there is an array of operations in a loop and I wanna wait for all of theme – Sepaloid

I don’t know what you mean. That’s what the above does, waits to fire the completion operation until they’re all done. – Shabbir

Re isAsynchronous: It is true that isAsynchronous is ignored when adding to a queue, but all of the isExecuting, isFinished and the associated KVO is required when adding operations that wrap some asynchronous process to a queue. That having been said, I believe it is best practice to set isAsynchronous to true when the operation is, indeed, asynchronous, to (a) reflect the reality of the operation and (b) to ensure it works whether it is used in a queue or just manually started. – Shabbir

Regarding the operation dependencies technique here, it looks like main is a class var on OperationQueue? Should that line be changed to OperationQueue.main.addOperation(completionOperation)? Does that have the same implications? – Ambroid

@Ambroid good catch. Both approaches work for the completion operation pattern (but barrier approach only works on the designated queue). Adjusted answer accordingly. – Shabbir

@Earflap maybe you can check my answer on this. – Gilda

Answer 2

A suitable solution is KVO

First before the loop add the observer (assuming queue is the OperationQueue instance)

queue.addObserver(self, forKeyPath:"operations", options:.new, context:nil)

Then implement

override func observeValue(forKeyPath keyPath: String?, of object: Any?, change: [NSKeyValueChangeKey : Any]?, context: UnsafeMutableRawPointer?) {
    if object as? OperationQueue == queue && keyPath == "operations" {
        if queue.operations.isEmpty {
            // Do something here when your queue has completed
            self.queue.removeObserver(self, forKeyPath:"operations")
        }
    } else {
        super.observeValue(forKeyPath: keyPath, of: object, change: change, context: context)
    }
}

Edit:

In Swift 4 it's much easier

Declare a property:

var observation : NSKeyValueObservation?

and create the observer

observation = queue.observe(\.operationCount, options: [.new]) { [unowned self] (queue, change) in
    if change.newValue! == 0 {
        // Do something here when your queue has completed
        self.observation = nil
    }
}

Since iOS13 and macOS15 operationCount is deprecated. The replacement is to observe progress.completedUnitCount.

Another modern way is to use the KVO publisher of Combine

var cancellable: AnyCancellable?

cancellable = queue.publisher(for: \.progress.completedUnitCount)
    .filter{$0 == queue.progress.totalUnitCount}
    .sink() { _ in 
       print("queue finished") 
       self.cancellable = nil           
    }

Contamination

the operationCount property was deprecated. Does that mean it shouldn't be used? – Eastereasterday

Answer 3

I use the next solution:

private let queue = OperationQueue()

private func addOperations(_ operations: [Operation], completionHandler: @escaping () -> ()) {
    DispatchQueue.global().async { [unowned self] in
        self.queue.addOperations(operations, waitUntilFinished: true)
        DispatchQueue.main.async(execute: completionHandler)
    }
}

Adularia

Answer 4

Set the maximum number of concurrent operations to 1

operationQueue.maxConcurrentOperationCount = 1

then each operation will be executed in order (as if each was dependent on the previous one) and your completion operation will execute at the end.

Panel

Just be careful here: this only holds if all operations in the queue have the same relative queuePriority and become ready in the order they were added to the serial queue. See the "Determine the Execution Order" section in the OperationQueue docs here – Marris

Answer 5

Code at the end of the queue refer to this link

NSOperation and NSOperationQueue are great and useful Foundation framework tools for asynchronous tasks. One thing puzzled me though: How can I run code after all my queue operations finish? The simple answer is: use dependencies between operations in the queue (unique feature of NSOperation). It's just 5 lines of code solution.

NSOperation dependency trick with Swift it is just easy to implement as this:

extension Array where Element: NSOperation {
/// Execute block after all operations from the array.
func onFinish(block: () -> Void) {
    let doneOperation = NSBlockOperation(block: block)
    self.forEach { [unowned doneOperation] in doneOperation.addDependency($0) }
    NSOperationQueue().addOperation(doneOperation)
}}

Sepaloid

Answer 6

My solution is similar to that of https://mcmap.net/q/756149/-wait-for-all-operations-in-queue-to-finish-before-performing-task, but I don't add the completionOperation in the main OperationQueue but into the queue itself. This works for me:

var a = [Int](repeating: 0, count: 10)

let queue = OperationQueue()

let completionOperation = BlockOperation {
    print(a)
}

queue.maxConcurrentOperationCount = 2
for i in 0...9 {
    let operation = BlockOperation {
        a[i] = 1
    }
    completionOperation.addDependency(operation)
    queue.addOperation(operation)
}

queue.addOperation(completionOperation)

print("Done 🎉")

Excrescence

Answer 7

I have met this problem, too. Well, instead of NSOperation, the GCD has already provide api: dispatch_group, for example:

dispatch_group_t myGroup = dispatch_group_create();

// task1
dispatch_group_enter(myGroup);
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
    // excute task1
    // ...

    // task1 completed，leave group
    dispatch_group_leave(myGroup);
});

// task2
dispatch_group_enter(myGroup);
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
    // excute task2
    // ...

    // task2 completed，leave group
    dispatch_group_leave(myGroup);
});

dispatch_group_notify(myGroup, dispatch_get_main_queue(), ^{
    // call back on all tasks c finished
    NSLog(@"all tasks are finished");
});

// continue other tasks.

Gilda