Making multiple coroutine API calls and waiting all of them

Question

so usually when you have to make different API calls and wait, you do something like this:

viewModelScope.launch {
    withContext(dispatcherProvider.heavyTasks) {
        val apiResponse1 = api.get1() //suspend function
        val apiResponse2 = api.get2() //suspend function

        if (apiResponse1.isSuccessful() && apiResponse2.isSuccessful() { .. }
    }
}

but what happens if I've to do multiple concurrent same API Calls with different parameter:

viewModelScope.launch {
    withContext(dispatcherProvider.heavyTasks) {
        val multipleIds = listOf(1, 2, 3, 4, 5, ..)
        val content = arrayListOf<CustomObj>()

        multipleIds.forEach { id ->
             val apiResponse1 = api.get1(id) //suspend function

             if (apiResponse1.isSuccessful()) {
                 content.find { it.id == id }.enable = true
             }
        }

        liveData.postValue(content)
    }
}

Problem with second approach is that it will go through all ids of multipleIds list and make async calls, but content will be posted probably before that. How can I wait all the responses from for each loop to be finished and only then postValue of the content to view?

Xerography

Maybe using async and awaiting on theme will help – Emplace

Answer 1

The preferred way to ensure a couple of asynchronous tasks have completed, is using coroutineScope. It will suspend until all child jobs, e.g. all calls to launch or async, have completed.

viewModelScope.launch {
    withContext(dispatcherProvider.heavyTasks) {
        val multipleIds = listOf(1, 2, 3, 4, 5, ..)
        val content = arrayListOf<CustomObj>()
        
        coroutineScope {
            multipleIds.forEach { id ->
                launch { // this will allow us to run multiple tasks in parallel
                    val apiResponse = api.get(id)
                    if (apiResponse.isSuccessful()) {
                        content.find { it.id == id }.enable = true
                    }
                }
           }
        }  // coroutineScope block will wait here until all child tasks are completed
        
        liveData.postValue(content)
    }
}

If you do not feel comfortable with this rather implicit approach, you can also use a more functional approach, mapping your ids to a list of Deferred using async and then await them all. This will also allow you to run all tasks in parallel but end up with a list of results in the correct order.

viewModelScope.launch {
    withContext(dispatcherProvider.heavyTasks) {
        val multipleIds = listOf(1, 2, 3, 4, 5, ..)
        val content = arrayListOf<CustomObj>()

        val runningTasks = multipleIds.map { id ->
                async { // this will allow us to run multiple tasks in parallel
                    val apiResponse = api.get(id)
                    id to apiResponse // associate id and response for later
                }
        }

        val responses = runningTasks.awaitAll()

        responses.forEach { (id, response) ->
            if (response.isSuccessful()) {
                content.find { it.id == id }.enable = true
            }
        }
      
        liveData.postValue(content)
    }
}

Quicken

What do you do if each Rest call is different with a different response? – Marquettamarquette

How about catching the errors? – Coverup

Answer 2

In order to get a concurrent behavior you need to start a new coroutine for each id. You can move multipleIds and content outside withContext block. Also you can post the result after withContext block since withContext is a suspending function so every coroutine created inside has to finish before posting the result.

viewModelScope.launch {
    val multipleIds = listOf(1, 2, 3, 4, 5, ..)
    val content = arrayListOf<CustomObj>()

    withContext(dispatcherProvider.heavyTasks) {
        multipleIds.forEach { id ->
            launch {
                val apiResponse = api.get(id) //suspend function
                if (apiResponse.isSuccessful()) {
                    content.find { it.id == id }?.enable = true
                }
            }
        }
    }

    liveData.value = content
}

Tecumseh

When I put break-point at liveData.value - it always works as expected, all list items are true (api.get(id)- have been called and response got), but when I run code with out break-point, all list items are false (logs show that api call was successful), meaning, that response came later. This does not work. – Xerography

Answer 3

Instead of forEach, go with map and do the same inside the { } block. Save result of map to a variable and post this variable.

Cenogenesis