How to assure that operations in an OperationQueue are finished one after another

Question

When performing operations that are dependent on each other OperationQueue could be used to assure they are executed in the correct order. However, would it also be possible to assure that operations are finished one after another?

Let's assume a method that is performed asynchronous and takes some time to finish:

public func performOperation(_ number: Int, success: @escaping (Int) -> Void)->Void {
        DispatchQueue(label: "operations").async {
            print("Operation #\(number) starts")
            usleep(useconds_t(1000-number*200)) // Block thread for some time
            success(number)
        }
}

The operations and dependencies are created as follows:

let operationQueue = OperationQueue.main
for operationNumber in 0..<4 { // Create operations as an example
    let operation = BlockOperation(block: {
        performOperation(operationNumber) { number in
            DispatchQueue.main.sync {
                print("Operation #\(number) finished")
            }
        }
    })
    operation.name = "Operation #\(operationNumber)"

    if operationNumber > 0 {
        operation.addDependency(operationQueue.operations.last!)
        // Print dependencies
        print("\(operation.name!) should finish after \(operation.dependencies.first!.name!)")
    }
    operationQueue.addOperation(operation)
}

With the following output:

Operation #1 should finish after Operation #0
Operation #2 should finish after Operation #1
Operation #3 should finish after Operation #2
Operation #0 starts
Operation #1 starts
Operation #2 starts
Operation #3 starts
Operation #0 finished
Operation #3 finished
Operation #2 finished
Operation #1 finished

This is clearly not correct. It seems that OperationQueue only assures that the operations are started in the right order (instead of finishing one after another). Although this could be performed using DispatchSemaphore, I was wondering whether it would also be possible with OperationQueue.

Churchly

You will have to create a subclass of NSOperation, override the asynchronous property, return true, override finished property and return true only when your asynchronous task done executing, can't write answer here as it would take a blog :) But I had asked question little complicated than yours you can use the code I have posted in my question :) have a look at #39201784 If found use full up vote the question and answer :D – Columbia

There's not a very common need to subclass NSOperation anymore – using a BlockOperation is simpler in most cases, and swapping to an equivalent NSOperation subclass would also not fix the issue in the code that is actually leading to non-deterministic finishing order of the operations. – Roseleeroselia

Answer 1

Operation dependencies are on finishing, not starting operations, so the system is behaving just as documented there. The issue is the DispatchQueue(label: "operations").async – your performOperation method exits right after you inside it asynchronously dispatch the sequence of print …; usleep …; success …, into a new dispatch queue created for every performOperation call. That sequence of print / sleep / success callback then gets executed on different threads of the worker thread pool that's managed by Grand Central Dispatch.

I think what you may be confused here is thinking that stating DispatchQueue(label: "operations") repeatedly would get you the same serial dispatch queue instance – this is not the case, you in fact create a new serial queue every call time.

As an aside, there's also no reason to create or dispatch to a serial dispatch queue inside your performOperation, as BlockOperation is already implemented such that the block gets executed concurrently on a GCD dispatch queue backing the OperationQueue (the concurrency is also possible to limit). What I would do in your case is construct a new OperationQueue with OperationQueue() (instead of using OperationQueue.main which dispatches work on the main queue), then asynchronously dispatch your success callbacks onto the main queue.

This slightly modified example shows you that the operation execution is indeed following the dependencies (I did not implement the above OperationQueue related suggestion it's arguably beside the point of the question you raised):

public func performOperation(_ number: Int, success: @escaping (Int) -> Void)->Void {
    print("Operation #\(number) starts")
    usleep(useconds_t(1000-(number*50))) // Block thread for some time
    success(number)
}

… 

let operationQueue = OperationQueue.main
for operationNumber in 0..<8 { // Create operations as an example
    let operation = BlockOperation(block: {
        self.performOperation(operationNumber) { number in
            print("Operation #\(number) finished")
        }
    })
    operation.name = "Operation #\(operationNumber)"

    if operationNumber > 0 {
        operation.addDependency(operationQueue.operations.last!)
        // Print dependencies
        print("\(operation.name!) should finish after \(operation.dependencies.first!.name!)")
    }
    operationQueue.addOperation(operation)
}

This will output…

Operation #1 should finish after Operation #0
Operation #2 should finish after Operation #1
Operation #3 should finish after Operation #2
Operation #4 should finish after Operation #3
Operation #5 should finish after Operation #4
Operation #6 should finish after Operation #5
Operation #7 should finish after Operation #6
Operation #0 starts
Operation #0 finished
Operation #1 starts
Operation #1 finished
Operation #2 starts
Operation #2 finished
Operation #3 starts
Operation #3 finished
Operation #4 starts
Operation #4 finished
Operation #5 starts
Operation #5 finished
Operation #6 starts
Operation #6 finished
Operation #7 starts
Operation #7 finished

Roseleeroselia

That's a clear explanation! However, I put the DispatchQueue(label: "operations") in the example method since in my real application Alamofire requests are performed which uses several dispatch queues by default (although I am not sure on what queue the completion handler is called). Would it still be possible to maintain the output order (see my updated questions)? – Churchly

To keep this question clear I removed my edit and asked a new question where I explicitly mentioned the use of Alamofire. I did not realize that the result would be so much different. Thanks for your help – Churchly

Answer 2

See this example to understand BlockOperation elegantly Example

Tympany