JavaScript: Asynchronous method in while loop

Question

I'm tackling a project that requires me to use JavaScript with an API method call. I'm a Java programmer who has never done web development before so I'm having some trouble with it.

This API method is asynchronous and it's in a while loop. If it returns an empty array, the while loop finishes. Otherwise, it loops. Code:

var done = true;

do
{
    async_api_call(
        "method.name", 
        { 
            // Do stuff.
        },
        function(result) 
        {
            if(result.error())
            {
                console.error(result.error());
            }
            else
            {
                // Sets the boolean to true if the returned array is empty, or false otherwise.
                done = (result.data().length === 0) ? true : false;
            }
        }
    );

} while (!done);

This doesn't work. The loop ends before the value of "done" is updated. I've done some reading up on the subject and it appears I need to use promises or callbacks because the API call is asynchronous, but I can't understand how to apply them to the code I have above.

Help would be appreciated!

Garges

Do you really need to use a while loop? The idea of async. programming is to avoid looping until something is done, but use the callback (function(result) in your case) to update the UI. – Clarance

Your async_api_call i already fired so it will pass, it won;t wait for the callback. Isn't that the purpose of async calls? Read this first #748675 maybe it will clarify the meaning of async functions. And because you cant update yet the done variable, !done is false and breaks the do while loop. – Formic

I need to loop async_api_call multiple times unfortunately. This is because the method in question only processes data in batches of 50, and there are thousands of items I need to process, so I need to continue looping it until all items are processed. – Garges

The important thing to understand is that the inner function (function(result)) will be executed at some later point in time, possibly long after the parent function finished executing; async_api_call does not block until the async action is complete and then executes the callback, it returns much earlier and executes the callback when the work is done. There are already some answers that help, just wanted to make that clear. – Zillah

can you use es6 here? – Hangeron

Just a tip. You can simply do: done = result.data().length === 0, instead of having that ? true : false after it. – Alecalecia

Answer 1

edit: see the bottom, there is the real answer.

I encourage you yo use the Promise API. Your problem can be solved using a Promise.all call:

let promises = [];
while(something){
    promises.push(new Promise((r, j) => {
        YourAsyncCall(() => r());
    });
}
//Then this returns a promise that will resolve when ALL are so.
Promise.all(promises).then(() => {
    //All operations done
});

The syntax is in es6, here is the es5 equivalent (Promise API may be included externally):

var promises = [];
while(something){
    promises.push(new Promise(function(r, j){
        YourAsyncCall(function(){ r(); });
    });
}
//Then this returns a promise that will resolve when ALL are so.
Promise.all(promises).then(function(){
    //All operations done
});

You can also make your api call return the promise and push it directly to the promise array.

If you don't want to edit the api_call_method you can always wrap your code in a new promise and call the method resolve when it finishes.

edit: I have seen now the point of your code, sorry. I've just realized that Promise.all will not solve the problem.

You shall put what you posted (excluding the while loop and the control value) inside a function, and depending on the condition calling it again.

Then, all can be wraped inside a promise in order to make the external code aware of this asynchronous execution. I'll post some sample code later with my PC.

So the good answer

You can use a promise to control the flow of your application and use recursion instead of the while loop:

function asyncOp(resolve, reject) {
    //If you're using NodeJS you can use Es6 syntax:
    async_api_call("method.name", {}, (result) => {
      if(result.error()) {
          console.error(result.error());
          reject(result.error()); //You can reject the promise, this is optional.
      } else {
          //If your operation succeeds, resolve the promise and don't call again.
          if (result.data().length === 0) {
              asyncOp(resolve); //Try again
          } else {
              resolve(result); //Resolve the promise, pass the result.
          }
      }
   });
}

new Promise((r, j) => {
    asyncOp(r, j);
}).then((result) => {
    //This will call if your algorithm succeeds!
});

/*
 * Please note that "(...) => {}" equivals to "function(...){}"
 */

Disc

if you aim for es6 then use async await. Tooo much levels here – Hangeron

@EduardJacko Yeah async/await is perfect for that stuff, but when the answer was created async/await I didn't know about it – Disc

actually his async_api_call accept callback. there is no way you could use async await anyway. sorry. – Hangeron

Answer 2

sigmasoldier's solution is correct, just wanted to share the ES6 version with async / await:

const asyncFunction = (t) => new Promise(resolve => setTimeout(resolve, t));

const getData = async (resolve, reject, count) => {

    console.log('waiting');
    await asyncFunction(3000);
    console.log('finshed waiting');

    count++;

    if (count < 2) {
        getData(resolve, reject, count);
    } else {
        return resolve();
    }
}

const runScript = async () => {
    await new Promise((r, j) => getData(r, j, 0));
    console.log('finished');
};

runScript();

Apartheid

beautiful, this guided me to what I needed to do. My particular case i needed to "loop" through async calls but each consecutive call depended on some data returned by the previous one. Adding some more variables and modifying count for my needs i got it to work. The only thing i dont quite fully understand is the necessity to wrap getData() itself within a promise, if getData itself returns a promise (since async func), why cant we just call that directly and expect the same result? – Hook

I think your are right @xunux. There is no need for the promise, since the async function should return a promise natively. – Apartheid

It's what would make sense to me, yet i needed to do it for it to work, i think it;s the passing down of the resolve and reject function from the original wrapping promise that is needed, cuz if i just did await getData() even though it returns a promise i cant pass it the resolve and reject needed to kinda end each loop – Hook

Answer 3

If you don't want to use recursion you can change your while loop into a for of loop and use a generator function for maintaining done state. Here's a simple example where the for of loop will wait for the async function until we've had 5 iterations and then done is flipped to true. You should be able to update this concept to set your done variable to true when your webservice calls have buffered all of your data rows.

let done = false;
let count = 0;
const whileGenerator = function* () {
    while (!done) {
        yield count;
    }
};

const asyncFunction = async function(){
    await new Promise(resolve => { setTimeout(resolve); });
};
const main = new Promise(async (resolve)=>{
    for (let i of whileGenerator()){
       console.log(i);
       await asyncFunction();

       count++;
       if (count === 5){
           done = true;
       }
    }
    resolve();
});
main.then(()=>{
    console.log('all done!');
});

Calcicole

Keep in mind, this solution is stateful. Meaning that you cannot share the state variables and generator across multiple for loops without potentially having issues. – Calcicole

Answer 4

Also you may try recursion solution.

function asyncCall(cb) {
// Some async operation
}

function responseHandler(result) {
    if (result.error()) {
        console.error(result.error());
    } else if(result.data() && result.data().length) {
        asyncCall(responseHandler);
    }
}

asyncCall(responseHandler);

Debonair

Answer 5

Here is a solution I came up with. Place this in an async function.

        let finished = false;
        const loop = async () => {
            return new Promise(async (resolve, reject) => {
                const inner = async () => {
                    if (!finished) {
                        //insert loop code here
                        if (xxx is done) { //insert this in your loop code after task is complete
                           finshed = true;
                           resolve();
                        } else {
                           return inner();
                        }
                    }
                }
                await inner();
            })
        }
        await loop();

Fruitless

Answer 6

If you don't want to use Promises you can restructure your code like so:

var tasks = [];
var index = 0;

function processNextTask()
{
    if(++index == tasks.length)
    {
        // no more tasks
        return;
    }

    async_api_call(
        "method.name", 
        { 
            // Do stuff.
        },
        function(result) 
        {
            if(result.error())
            {
                console.error(result.error());
            }
            else
            {
                // process data
                setTimeout(processNextTask);
            }
        }
    );
}

Pantin

Answer 7

Your loop won't work, because it is sync, your async task is async, so the loop will finish before the async task can even respond. I'd reccomend you to use Promises to manage async tasks:

//first wrapping your API into a promise
var async_api_call_promise = function(methodName, someObject){
    return new Promise((resolve, reject) => {
        async_api_call(methodName, someObject, function(result){
            if(result.error()){ 
                reject( result.error() ) 
            }else{
                resolve( result.data() )
            }
        });
    })
}

now to your polling code:

//a local utility because I don't want to repeat myself
var poll = () => async_api_call_promise("method.name", {/*Do stuff.*/});

//your pulling operation
poll().then(
    data => data.length === 0 || poll(),  //true || tryAgain
    err => {
        console.error(err);
        return poll();
    }
).then((done) => {
    //done === true
    //here you put the code that has to wait for your "loop" to finish
});

Why Promises? Because they do state-management of async operations. Why implement that yourself?

Dunston

Answer 8

  let taskPool = new Promise(function(resolve, reject) {
    resolve("Success!");
  });
  let that = this;
  while (index < this.totalPieces) {
    end = start + thisPartSize;
    if (end > filesize) {
      end = filesize;
      thisPartSize = filesize - start;
    }
    taskPool.then(() => {
      that.worker(start, end, index, thisPartSize);
    });
    index++;
    start = end;
  }

Transported

Answer 9

I've actually switched to undwinding a while loop into a for loop instead of my other answer to use a generator. It's a bit cleaner and more straight forward as to what is happening without the need for extra state.

const asyncFunction = async ()=>{
  await new Promise(resolve => { setTimeout(resolve); });
}

(async()=>{
  let i = 0;
  for (;;){
    await asyncFunction();
    
    console.log(i);
    // you have to manually perform your while condition in the loop
    if (i === 5) {
      // condition is true so break the infinite for loop
      break;
    }
    i++;
  }
  console.log('done');
})();

Calcicole