implementing a completion handler for an asynchronous dispatch queue which calls asynchronous methods

Question

I have a function performSync which runs through an array and for each item in this array, I am calling a function which in itself contains an async alamofire request. I need to be able to tell when this outer function has completed running all functions within the for loop, so I need to add a completion handler. I am unsure how to implement this.

Also, in my for loop, I have used .userinitiated in order to try and not block the ui thread, however the thread is being blocked. I also tried using the commented out concurrent queue method, but that also blocked the UI.

Code below:

public class Sync {
        public class func onPerformSync(finished: () -> Void){

        let syncList = ['Process1', 'Process2', 'Process3']
        //let concurrentQueue = DispatchQueue(label: "concurrentQueue", attributes: .concurrent)
        for item in syncList {
             //concurrentqueue.async
             DispatchQueue.global(qos: .background).async {
                 let className = "appname."+item
                 let ClassObj = NSClassFromString(className)! as! Helper_Base.Type
                 ClassObj.doSync()
             }
            //multiple classes extend Helper_Base and override the  doSync func  
            //I realise there may be a swiftier way to call these doSync methods rather than instantiating from a string and overriding the base class 
           //doSync and I'd welcome advice on it!
        }
        //calling finished here is fine if I use a synchronous dispatchQueue but where do i place this line since I need my queue to be asynchronous?
        finished()
    }
}

open class Process1 : Helper_Base {
    override open class func doSync(){
        let time = Int64(NSDate().timeIntervalSince1970 * 1000)
        repeatBlock(time: time)
    }

    open class func repeatBlock(time : Int64){
        let parameters : [String : String] = [
            "x" : time
        ]
        var continueSync : Bool = false
        DispatchQueue.global(qos: .background).async {
        Alamofire.request(url, method: .post, parameters: parameters, encoding: URLEncoding.default)
            .response { response in
                if let data = response.data, let utf8Text = String(data: data, encoding: .utf8) {
                    guard let utf8TextDecoded = utf8Text.fromBase64() else {
                        return
                    }

                    let error = response.error
                    if error == nil
                    {
                        do {
                            let json = try JSONSerializer.toDictionary(utf8TextDecoded)
                            let more = json["more"] as! Bool

                            continueSync = more
                        }
                    }
                }
                if continueSync {
                    let time = Int64(NSDate().timeIntervalSince1970 * 1000)
                    DispatchQueue.global(qos: .background).async {
                        repeatBlock(time: time)
                    }
                }
                else{
                   finishSync()
                }
        }

    }
    }

    open class func finishSync(){
        //other stuff
    }
}


Sync.onPerformSync(){
    print("we're done syncing")
}

Introduction

Answer 1

A couple of observations:

1. In answer to your main question, how to have finishSync inform its caller that it was done, you'd probably refactor doSync to take a completion handler. This would (because of the fact that it's doing recursive network calls) suggest that you might move to instance methods and save the completionHandler:

   public class HelperClass {
   
       private var completionHandler: (() -> Void)?
   
       open func doSync(item: String, completionHandler: @escaping () -> Void) {
           guard self.completionHandler == nil else {        // make sure the existing completionHandler is nil
               fatalError("Either use another instance or let this one finish first")
           }
   
           self.completionHandler = completionHandler        // save the completion handler
   
           repeatBlock()                                     // start your recursive code
       }
   
       open func repeatBlock() {                             // do your recursive stuff
           // do something
       }
   
       open func finishSync(){
           // other stuff
   
           completionHandler?()                              // call the saved completion handler
           completionHandler = nil                           // and discard it, removing any strong references, if any
       }
   }
   

2. This only then begs the question of how you make sure that top level for loop for process1, process2, etc., doesn't run these loops concurrently. Frankly, you generally would want it to run concurrently (because you pay huge performance penalty to run requests sequentially), but if you don't, you'd either have to wrap this in yet another recursive requesting process or wrap the whole thing in an asynchronous Operation custom subclass.

3. Regarding why your UI is blocked, that's less clear. If anything, you don't even need to dispatch the doSync to some global queue, because Alamofire is asynchronous. There's no point in dispatching a call that is already asynchronous to a background queue.

   The only thing that looks suspicious is the the recursive call from repeatBlock back to repeatBlock. If the Alamofire call doesn't run asynchronously (i.e. it returned cached results or there's some error), you could theoretically spin on the queue that Alamofire is using. I'd suggest dispatching that recursive call to repeatBlock from within repeatBlock with async, to avoid any potential problems.

   The other thing you can do is to supply the queue parameter to the Alamofire request, ensuring it's not using the main queue. Left to its own devices, it calls its completion handlers on the main queue (which is generally quite useful, but could cause problems in your scenario). I would suggest trying supplying a non-main queue as the queue parameter of request.

   If it's still blocking, I'd suggest either running it through the system trace of Instruments and see what blocking calls there are on the main queue. Or you can also just run the app, pause it while the UI is frozen, and then look at the stack trace for the main thread, and you might see where it's blocking.

   Finally, we must contemplate the possibility that the blocking of the main queue does not rest in the above code. Assuming that you're not suffering from degenerate situation where Alamofire is calling its completion handler immediately, the above code seems unlikely to block the main queue. The above diagnostics should confirm this, but I might suggest you broaden your search to identify other things that would block the main queue:

   • Any sync calls from a serial queue (such as the main queue) to a serial queue are a likely problem.
   • Or any locks, semaphores, or dispatch group waits are also likely candidates.
   • You have paths where you neither recursively call repeatBlock nor call finishSync ... are you sure the main queue is blocked and it's not just a matter that it's never calling finishSync in some paths of execution.
      

   Bottom line, make sure the problem actually rests in blocking the main thread in the above code. I suspect it might not.

To close with a bit of unsolicited advice, and with no offense intended, we should acknowledge that there is a hint of a code smell in this flurry of network requests. If you want to return more data, change the web service API to return more data ... this repeated, systematic fetching of additional data (especially when done sequentially) is horribly inefficient. The network latency is going to make overall performance (even after you solve this main queue blocking problem) really suffer.

Impolitic

thank you for your detailed answer... 1. I have implemented the completion handlers but instead of switching to instance methods I have passed the completion handler from doSync to repeatBlock to finishSync. I've edited my question to reveal why I didn't switch to instance methods. 2. I would like the for loop to run concurrently but I'd like to somehow tell when all processes within the for loop have hit their completion handler. What would be the best approach to this in your opinion? – Introduction

3. I believe I am now dispatching that recursive call to repeatBlock from within repeatBlock. I also wrapped the the alamofire request in the same way but I'm not sure if this is what you meant. The UI was still blocking after these updates but I noticed an error in the logs which may be the reason for the UI block 'uncaught exception in notifier thread: N5realm10LogicErrorE: Bad version number'. The code left out in the completionhandler of alamofire involved addding entries to a realm DB – Introduction

4. Re. the advice, I am always appreciative, however I cannot change what is passed back from the network calls. Note it is only in rare cases that the repeatBlock is called multiple times. – Introduction

Answer 2

I have had similar problems - and I used a global variable (a singleton) to increment a counter for each new request that was started and then decrement that counter in each completion handler.

If you get to myQueueCounter == 0 then you have finished, and can call the doneSyncing method

if there is a risk that some of the queues could complete before all of them have been initiated, then you might need an additional variable for everythingHasStarted = true

Lindesnes