Synchronise multiple web service calls in serial order in swift

Question

I am hitting a web service url 10 times and getting the response. I am using Alamofire and SwiftyJSON. This is my controller code

class ViewController: UIViewController {

    let dispatchGroup = DispatchGroup()

    var weatherServiceURL = "http://samples.openweathermap.org/data/2.5/weather?q=London,uk&appid=b6907d289e10d714a6e88b30761fae22"

    override func viewDidLoad() {
        super.viewDidLoad()
        start()
    }

    func start() {
        weatherService()
        dispatchGroup.notify(queue: .main) {
            print("All services complete")
        }
    }

    func weatherService() {
        for i in 1...10 {
            dispatchGroup.enter()
            APIManager.apiGet(serviceName: self.weatherServiceURL, parameters: ["counter":i]) { (response:JSON?, error:NSError?, count:Int) in
                if let error = error {
                    print(error.localizedDescription)
                    return
                }
                guard let response = response else { return }
                print("\n\(response) \n\(count) response\n")
                self.dispatchGroup.leave()
            }
        }
    }
}

This is my Service Handler class code

class APIManager: NSObject {

    class func apiGet(serviceName:String,parameters: [String:Any]?, completionHandler: @escaping (JSON?, NSError?, Int) -> ()) {
        Alamofire.request(serviceName, method: .get, parameters: nil, encoding: URLEncoding.default, headers: nil).responseJSON { (response:DataResponse<Any>) in

            switch(response.result) {
            case .success(_):
                if let data = response.result.value{
                    let json = JSON(data)
                    completionHandler(json,nil, parameters!["counter"] as! Int)
                }
                break

            case .failure(_):
                completionHandler(nil,response.result.error as NSError?, parameters!["counter"] as! Int)
                break
            }
        }
    }
}

I am sending a counter key with the index of for loop just to keep the track of response of which index is coming back. But the response is not coming in serial order. We can expect 3rd response before the 2nd and 1st response. This is because the API call with APIManager.apiGet function call is asynchronous and is escaping and therefore continuing the for loop.

Also I used the dispatchQueue

let dispatchQueue = DispatchQueue(label: "com.test.Queue", qos: .userInteractive)

and converted the function as:

func weatherService() {
    for i in 1...10 {
        dispatchGroup.enter()
        dispatchQueue.async {
            APIManager.apiGet(serviceName: self.weatherServiceURL, parameters: ["counter":i]) { (response:JSON?, error:NSError?, count:Int) in
                if let error = error {
                    print(error.localizedDescription)
                    return
                }
                guard let response = response else { return }
                print("\n\(response) \n\(count) response\n")
                self.dispatchGroup.leave()
            }
        }
    }
}

Same result as the service calling code is asynchronous. If we make

dispatchQueue.sync {
   //service call 
}

then also we will not get the response in serial order since the networking call in async and dispatchQueue assumes the task is completed.

Condition is to hit the service in async manner only without freezing the UI. If I hit the service is synchronous manner, then I get my desired result. But blocking main thread is not at all acceptable.

I can manage this thing using array or some global bool variables, but I don't want to use them. Is there any other way I can get response in serial order in which it is called? Any help or hint is appreciated.

Cringe

Nothing in your question indicates why it is a problem that the responses might complete in a different order. In fact, your question doesn't show any use of any of the responses other than a simple print. – Smoothbore

Yes, I was doing a simple test and not using the response in completion. This question was just to ensure that services should gracefully execute in serial order wrt their response asynchronously which is not happening. There are some ways in which I can ensure the serial response like calling other service in completion of 1st but I want to know whether it can be done without that approach or not – Cringe

Answer 1

Solution: Use DispatchSemaphores and a DispatchQueue

Rather than saving the closures, I decided to wrap everything up in a dispatch queue and use semaphores inside it

//Create a dispatch queue 
let dispatchQueue = DispatchQueue(label: "myQueue", qos: .background)

//Create a semaphore
let semaphore = DispatchSemaphore(value: 0)

func weatherService() {

    dispatchQueue.async {
        for i in 1...10 {
            APIManager.apiGet(serviceName: self.weatherServiceURL, parameters: ["counter":i]) { (response:JSON?, error:NSError?, count:Int) in
                if let error = error {
                    print(error.localizedDescription)
                    self.semaphore.signal()
                    return
                }
                guard let response = response else { 
                    self.semaphore.signal()
                    return 
                }

                print("\(count) ")

                //Check by index, the last service in this case
                if i == 10 {
                    print("Services Completed")
                } else {
                    print("An error occurred")
                }

                // Signals that the 'current' API request has completed
                self.semaphore.signal()
            }

            // Wait until the previous API request completes
            self.semaphore.wait()
        }
    }
    print("Start Fetching")
}

Output is this always

Cringe

Can I cancel this operation in midway, say at 5 or on a button click. – Stravinsky

@ShivamTripathi There is no way to stop semaphore.There is no "cancel semaphore wait" operation other than to signal it. To stop a semaphore on buttonClick, just signal the semaphore and take a bool variable which will be also present in your api execution code. That variable will check whether to execute the for loop further. So, its like, if semaphoreStopped { return } from the function where semaphoreStopped is a boolean which will be true on button click – Cringe

@RajanMaheshwari Reference to your answer can I ask one question that relates to DispatchGroup and DispatchQueue ? – Septa

@Septa Yes Please – Cringe

Answer 2

Idea

• index1 - index in your loop, when you create closure
• index2 - index of executed operation in container

You need to create container with closures. This container will save all closures. Container will check if index1 == index2 run all operations before index1 and after if index1 + 1 > exist.

So, this container will check the order of received closures and run closures one by one in ascending order.

Details

Xcode 9.4.1, Swift 4.1

Container

class ActionsRunController {

    typealias Func = ()->()
    private var actions: [Int: Func] = [:]
    private var dispatchSemaphore = DispatchSemaphore(value: 1)
    private var firstIndex = 0
    private var lastIndex = 0

    func add(at index: Int, action: Func?) {
        dispatchSemaphore.wait()
        actions[index] = action
        if lastIndex == index {
            while (actions[firstIndex] != nil) {
                actions[firstIndex]?()
                actions[firstIndex] = nil
                firstIndex += 1
            }
            lastIndex = firstIndex
        }
        dispatchSemaphore.signal()
    }
}

Full Code

Do no forget to add code of the Container here

import UIKit
import Alamofire
import SwiftyJSON

class ViewController: UIViewController {

    let dispatchGroup = DispatchGroup()

    var weatherServiceURL = "http://samples.openweathermap.org/data/2.5/weather?q=London,uk&appid=b6907d289e10d714a6e88b30761fae22"

    override func viewDidLoad() {
        super.viewDidLoad()
        start()
    }

    func start() {
        weatherService()
        dispatchGroup.notify(queue: .main) {
            print("All services complete")
        }
    }

    func weatherService() {
        for i in 0...9 {
            dispatchGroup.enter()
            APIManager.apiGet(serviceName: self.weatherServiceURL, counter: i) { (response:JSON?, error:NSError?, count:Int) in
                if let error = error {
                    print(error.localizedDescription)
                    return
                }
                //guard let response = response else { return }
                print("[executed] action \(count)")
                self.dispatchGroup.leave()
            }
        }
    }
}

class APIManager: NSObject {

    private static let actionsRunController = ActionsRunController()

    class func apiGet(serviceName:String, counter:  Int, completionHandler: @escaping (JSON?, NSError?, Int) -> ()) {
        Alamofire.request(serviceName, method: .get, parameters: nil, encoding: URLEncoding.default, headers: nil).responseJSON { (response:DataResponse<Any>) in

            //print("[created] action \(counter)")
            switch(response.result) {
            case .success(_):
                if let data = response.result.value{
                    let json = JSON(data)
                    actionsRunController.add(at: counter) {
                        completionHandler(json, nil, counter)
                    }
                }
                break

            case .failure(_):
                actionsRunController.add(at: counter) {
                    completionHandler(nil,response.result.error as NSError?, counter)
                }
                break
            }
        }
    }
}

Result

Astounding

This can do it, but is it good to use semaphores? Or if we can just keep a track of closures in an array in Controller itself and return that array when all services are completed? – Cringe

Yes, this is good for smartphones. Yes, if you want to wait when all connection will finish and then run completion block you will spend more users time. Because internet connections can be slow. Using this solution you will get you parts of servers data and you can display this parts. – Astounding

@RajanMaheshwari Whether or not semaphores is the current "waiting" object to use here is up for debate, but works for me. To address your question about tracking closures, I ran into a strange issue when trying to update YouTube video metadata with the YouTube API where tracking closures would not work, but using Semaphores would. Just FYI. – Regenaregency

How can I cancel rest of requests after one failure case? – Hosiery

@Hosiery this is complicated. E.g. you have the requests [a,b,c]. You want to cancel b,c if a failed, but you start all of them simultaneously. that means, that you will have to keep [a,b,c] (referrals to the requests) in common space to have access to [a,b,c] from any single request. And when a failed, in completion block of the a you will have to cancel b,c. You will have to implement this canceling logic in each request and [a,b,c] must be thread safe. So, this is not solution. This is only idea, direction where to find the answer of your question. – Astounding

Answer 3

The simplest way to get the api calls made in order is to perform the "next" call in the completion handler of the previous, rather than using a for loop outside the api calls.

func weatherService(counter: Int = 1, maxCount: Int = 10) {
    guard counter <= maxCount else {
        return
    }
    dispatchGroup.enter()
    APIManager.apiGet(serviceName: self.weatherServiceURL, parameters: ["counter":i]) { (response:JSON?, error:NSError?, count:Int) in
            self.weatherService(counter: counter+1, maxCount: maxCount)
            if let error = error {
                print(error.localizedDescription)
                self.dispatchGroup.leave()
                return
            }
            guard let response = response else {
                self.dispatchGroup.leave()
                return 
            }
            print("\n\(response) \n\(count) response\n")
            self.dispatchGroup.leave()
        }
    }
}

I would advise against this though unless there's is some dependency on order (i.e. call number 2 needs information from the result of call 1) because it will take longer than parallel requests.

It would be much better to handle the fact that results may return out of order.

Also, when using a dispatch group you need to ensure that you call dispatchGroup.leave in all cases where the code completes; in your case you are not doing so in the case where an error occurs. This would result in the dispatchGroup.notify never firing if an error occurs in one or more requests.

Extrude

Yes, the dispatchGroup.leave was not there in all cases. I should put in all. That's right. Regarding the serial order, I believe holding the closure in array would be one of the solution as discussed in Vasily's answer. But it may result in delays as we are using semaphore wait there.? – Cringe

The delays come from serialising the requests. Unless you need the requests to execute in a particular order (because a subsequent request depends on the result of an earlier request) it is better to just fire off all of the requests and deal with the out of order responses. – Extrude

Answer 4

code that can be used easily in multiple places:

extension Array where Element:Equatable {
func syncedExecution(execute:@escaping ((Element, @escaping (() -> Void)) -> ()), completion: (() -> Void)? = nil) {
    let dispatchQueue = DispatchQueue(label: UUID().uuidString, qos: .background)
    let dispatchSemaphore = DispatchSemaphore(value: 0)
    dispatchQueue.async {
        for item in self {
            execute(item) {
                dispatchSemaphore.signal()
                if item == last {
                    completion?()
                }
            }
            dispatchSemaphore.wait()
        }
    }
}
}

usage:

itemsArr. syncedExecution { item, signal in
    item.someApiRequest { success in
        // do something
        signal()
    }
} completion: {
        // all tasks completed
    }

Thermotaxis