Waiting for more than one concurrent await operation
Asked Answered
M

4

57

How can I change the following code so that both async operations are triggered and given an opportunity to run concurrently?

const value1 = await getValue1Async();
const value2 = await getValue2Async();
// use both values

Do I need to do something like this?

const p1 = getValue1Async();
const p2 = getValue2Async();
const value1 = await p1;
const value2 = await p2;
// use both values
Melodramatic answered 23/10, 2017 at 12:24 Comment(7)
The lower code block would do what you need. Alternatively use Kai's solution.Curmudgeon
Thank you. Side question: will the following force waiting for both (and discarding the results) await p1 && await p2?Melodramatic
Interesting question if p1 is a Promise<boolean> that resolves to false. Will it short-circuit?Curmudgeon
@Florian: Yes, it will (short-circuit), which isn't a good thing. :-) Ben: No, it won't (necessarily wait for both; as Florian points out, if the first resolves to a falsy value, it won't wait for the second at all, and so you may get an unhandled rejection error [if p2 rejects]). You'll also get one if both promises reject. I've updated my answer to address this...Allocation
Close to duplicate of this question from a while back - but I prefer to keep this since A) async/await is a lot more common than generators now and B) this is pretty simply phrased.Icing
@BenjaminGruenbaum Another possible duplicate: Any difference between await Promise.all() and multiple await?Vasily
Does this answer your question? Call async/await functions in parallelQuetzalcoatl
A
80

TL;DR

Don't use the pattern in the question where you get the promises, and then separately wait on them; instead, use Promise.all (at least for now):

const [value1, value2] = await Promise.all([getValue1Async(), getValue2Async()]);

While your solution does run the two operations in parallel, it doesn't handle rejection properly if both promises reject.

Details:

Your solution runs them in parallel, but always waits for the first to finish before waiting for the second. If you just want to start them, run them in parallel, and get both results, it's just fine. (No, it isn't, keep reading...) Note that if the first takes (say) five seconds to complete and the second fails in one second, your code will wait the full five seconds before then failing.

Sadly, there isn't currently await syntax to do a parallel wait, so you have the awkwardness you listed, or Promise.all. (There's been discussion of await.all or similar, though; maybe someday.)

The Promise.all version is:

const [value1, value2] = await Promise.all([getValue1Async(), getValue2Async()]);

...which is more concise, and also doesn't wait for the first operation to complete if the second fails quickly (e.g., in my five seconds / one second example above, the above will reject in one second rather than waiting five). Also note that with your original code, if the second promise rejects before the first promise resolves, you may well get a "unhandled rejection" error in the console (you do currently with Chrome v61; update: more recent versions have more interesting behavior), although that error is arguably spurious (because you do, eventually, handle the rejection, in that this code is clearly in an async function¹ and so that function will hook rejection and make its promise reject with it) (update: again, changed). But if both promises reject, you'll get a genuine unhandled rejection error because the flow of control never reaches const value2 = await p2; and thus the p2 rejection is never handled.

Unhandled rejections are a Bad Thing™ (so much so that soon, Node.js will abort the process on truly unhandled rejections, just like unhandled exceptions — because that's what they are), so best to avoid the "get the promise then await it" pattern in your question.

Here's an example of the difference in timing in the failure case (using 500ms and 100ms rather than 5 seconds and 1 second), and possibly also the arguably-spurious unhandled rejection error (open the real browser console to see it):

const getValue1Async = () => {
  return new Promise(resolve => {
    setTimeout(resolve, 500, "value1");
  });
};
const getValue2Async = () => {
  return new Promise((resolve, reject) => {
    setTimeout(reject, 100, "error");
  });
};

// This waits the full 500ms before failing, because it waits
// on p1, then on p2
(async () => {
  try {
    console.time("separate");
    const p1 = getValue1Async();
    const p2 = getValue2Async();
    const value1 = await p1;
    const value2 = await p2;
  } catch (e) {
    console.error(e);
  }
  console.timeEnd("separate");
})();

// This fails after just 100ms, because it doesn't wait for p1
// to finish first, it rejects as soon as p2 rejects
setTimeout(async () => {
  try {
    console.time("Promise.all");
    const [value1, value2] = await Promise.all([getValue1Async(), getValue2Async()]);
  } catch (e) {
    console.timeEnd("Promise.all", e);
  }
}, 1000);
Open the real browser console to see the unhandled rejection error.

And here we reject both p1 and p2, resulting in a non-spurious unhandled rejection error on p2:

const getValue1Async = () => {
  return new Promise((resolve, reject) => {
    setTimeout(reject, 500, "error1");
  });
};
const getValue2Async = () => {
  return new Promise((resolve, reject) => {
    setTimeout(reject, 100, "error2");
  });
};

// This waits the full 500ms before failing, because it waits
// on p1, then on p2
(async () => {
  try {
    console.time("separate");
    const p1 = getValue1Async();
    const p2 = getValue2Async();
    const value1 = await p1;
    const value2 = await p2;
  } catch (e) {
    console.error(e);
  }
  console.timeEnd("separate");
})();

// This fails after just 100ms, because it doesn't wait for p1
// to finish first, it rejects as soon as p2 rejects
setTimeout(async () => {
  try {
    console.time("Promise.all");
    const [value1, value2] = await Promise.all([getValue1Async(), getValue2Async()]);
  } catch (e) {
    console.timeEnd("Promise.all", e);
  }
}, 1000);
Open the real browser console to see the unhandled rejection error.

In a comment you've asked:

Side question: will the following force waiting for both (and discarding the results) await p1 && await p2?

This has the same issues around promise rejection as your original code: It will wait until p1 resolves even if p2 rejects earlier; it may generate an arguably-spurious (update: or temporary) unhandled rejection error if p2 rejects before p1 resolves; and it generates a genuine unhandled rejection error if both p1 and p2 reject (because p2's rejection is never handled).

Here's the case where p1 resolves and p2 rejects:

const getValue1Async = () => {
  return new Promise(resolve => {
    setTimeout(resolve, 500, false);
  });
};
const getValue2Async = () => {
  return new Promise((resolve, reject) => {
    setTimeout(reject, 100, "error");
  });
};

(async () => {
  try {
    const p1 = getValue1Async();
    const p2 = getValue2Async();
    console.log("waiting");
    await p1 && await p2;
  } catch (e) {
    console.error(e);
  }
  console.log("done waiting");
})();
Look in the real console (for the unhandled rejection error).

...and where both reject:

const getValue1Async = () => {
  return new Promise((resolve, reject) => {
    setTimeout(reject, 500, "error1");
  });
};
const getValue2Async = () => {
  return new Promise((resolve, reject) => {
    setTimeout(reject, 100, "error2");
  });
};

(async () => {
  try {
    const p1 = getValue1Async();
    const p2 = getValue2Async();
    console.log("waiting");
    await p1 && await p2;
  } catch (e) {
    console.error(e);
  }
  console.log("done waiting");
})();
Look in the real console (for the unhandled rejection error).

¹ "...this code is clearly in an async function..." That was true in 2017 when this question and answer were written. Since then, top-level await happened/is happening.

Allocation answered 23/10, 2017 at 12:32 Comment(12)
Kai was there first (thanks), but this is the more complete answer.Melodramatic
@Ben: There's an important difference between yours and Promise.all that I've just edited to call out, FYI.Allocation
"(so much so that soon, NodeJS will abort the process on unhandled rejections, just like unhandled exceptions — because that's what they are)" - the deprecation warning wording is unfortunate and I regret it - but we will never kill node on the code above - we will: A) Make unhandled rejections GC based B) warn on really long pending operations that GC missed, probably C) only ever kill Node.js if we can prove a rejection is unhandled (the first case). I realize the situation is confusing and I apologize for it - we'll do better.Icing
That said, this is definitely the right approach and using Promise.all in order to do this. Just a nit: there is never an unhandled rejection in the above code or OPs. The only reason it logs that is because our code for unhandled rejection detection isn't 100% perfect and relies on promises typically handling their rejections in a timely manner. In case anyone cares - here is a talk I gave about adding themIcing
@BenjaminGruenbaum: Thanks for that. I can see that for the case where p2 rejects before p1 resolves (the code handles the rejection, just after a delay), but surely both the OP's code and await p1 && await p2; result in an unhandled rejection if both p1 and p2 reject...? Nothing ever catches the rejection from p2... (I'll wait to correct the answer until you have a chance to reply.)Allocation
In await p1 && await p2 if both p1 and p2 reject then p2 is an unhandled rejection (and GC based detection will still kill the process rightfully). I was only talking about the case p2 rejects while p1 is still pending.Icing
@BenjaminGruenbaum: Great, thanks! When you say "our code" above, you're talking about the Node project? I just ask because Chrome also issues an unhandled notification notice and so also seems not to be using GC-based unhandled rejection handling, if you happen to have any insight into the handling of this at a V8 level. (I didn't find a V8 issue for it.)Allocation
@T.J.Crowder "our code" in this case is the Node project. In particular this is an area of the code I've been involved in - sorry for the ambiguity. Here is how we do it: github.com/nodejs/node/blob/master/lib/internal/process/… - There is github.com/nodejs/node/pull/15126 and github.com/nodejs/node/pull/15335 about ongoing work. In Chrome, you can see V8 bindings at github.com/nwjs/chromium.src/blob/… which is run at ProcessQueue after a task.Icing
That is, no one does "GC based" yet (firefox did at one point, not sure if they still do) - BridgeAR's PR shows the approach we're considering right now. Zones might also be an interesting idea.Icing
"...although that error is arguably spurious (because you do, eventually, handle the rejection)" Where is that handling happening?Sierrasiesser
@Sierrasiesser - It's really subtle and I shouldn't have just had that throw-away comment: The code in the question was (at the time) clearly in an async function. So that function will "handle" the rejection in that it will consume it and reject the promise the function returned. Naturally, something has to consume that function's promise and handle rejections from it (in my example, I'm doing it within the function, but)...Allocation
@Sierrasiesser - You may find this interesting as well...Allocation
P
10

I think this should work:

 const [value1, value2] = await Promise.all([getValue1Async(),getValue2Async()]);

A more verbose example is below in case it helps in understanding:

const promise1 = async() => {
  return 3;
}

const promise2 = async() => {
  return 42;
}

const promise3 = async() => {
  return 500;
  // emulate an error
  // throw "something went wrong...";
}

const f1 = async() => {

  try {
    // returns an array of values
    const results = await Promise.all([promise1(), promise2(), promise3()]);
    console.log(results);
    console.log(results[0]);
    console.log(results[1]);
    console.log(results[2]);

    // assigns values to individual variables through 'array destructuring'
    const [value1, value2, value3] = await Promise.all([promise1(), promise2(), promise3()]);

    console.log(value1);
    console.log(value2);
    console.log(value3);

  } catch (err) {
    console.log("there was an error: " + err);
  }

}

f1();
Pliny answered 23/10, 2017 at 12:26 Comment(1)
I got your idea. IMHO, it should work :). Sorry for my careless confirmationPliny
A
1

Use .catch() and Promise.all()

Make sure you handle rejections correctly and you can safely use Promises.all() without facing unhandled rejections. (Edit: clarification per discussion: not the Error unhandled rejection but simply rejections that are not being handled by the code. Promise.all() will throw the first promise rejection and will ignore the rest).

In the example below an array of [[error, results], ...] is returned to allow ease of processing results and/or errors.

let myTimeout = (ms, is_ok) =>
  new Promise((resolve, reject) => 
    setTimeout(_=> is_ok ? 
                   resolve(`ok in ${ms}`) :
                   reject(`error in ${ms}`),
               ms));

let handleRejection = promise => promise
  .then((...r) => [null, ...r])
  .catch(e => [e]); 

(async _=> {
  let res = await Promise.all([
    myTimeout(100, true),
    myTimeout(200, false),
    myTimeout(300, true),
    myTimeout(400, false)
  ].map(handleRejection));
  console.log(res);
})();

You may throw from within a catch() to stop waiting for all (and discard the results of the rest), however - you may only do it once per try/catch blocks so a flag has_thorwn need to be maintained and checked to make sure no unhandled errors happens.

let myTimeout = (ms, is_ok) =>
  new Promise((resolve, reject) =>
    setTimeout(_=> is_ok ?
                   resolve(`ok in ${ms}`) :
                   reject(`error in ${ms}`),
               ms));

let has_thrown = false;

let handleRejection = promise => promise
  .then((...r) => [null, ...r])
  .catch(e => {
    if (has_thrown) {
      console.log('not throwing', e);
    } else {
      has_thrown = 1;
      throw e;
    }
  });

(async _=> {
  try {
    let res = await Promise.all([
      myTimeout(100, true),
      myTimeout(200, false),
      myTimeout(300, true),
      myTimeout(400, false)
    ].map(handleRejection));
    console.log(res);
  } catch(e) {
    console.log(e);
  }
  console.log('we are done');
})();
Averyaveryl answered 17/11, 2018 at 10:11 Comment(12)
I think this doesn't really answer the question, and catch in this location really is not necessary to avoid unhandled rejections. Also that [error, results] pattern is a really bad ideaVasily
@Vasily - without handling rejections correctly there is no way to avoid that unhandled promise rejection (which is heavily discussed in the accepted answer) that will (in the future) terminate node process. The pattern [err, results] is just an example of how to pass and handle multiple errors at the end.Averyaveryl
@Bergi, about answering the question: Promise.all() is not answering? In addition, "...and given an opportunity to run concurrently" - with out handling correctly, if one is rejected the others are not given the opportunity to return result.Averyaveryl
No, you don't need .catch() on the individual promises, Promise.all is totally capable of preventing unhandled rejections on them (as discussed in the accepted answer) by itself.Vasily
"if one is rejected the others are not given the opportunity to return result" - that's a totally different questionVasily
@bergi, where in the accepted answer you see Promise.all() preventing unhandled rejections? In fact, in every example there you will see unhandled rejection (open the console to see "Uncaught (in promise) error"). Please see #30363233. The fact that there is more specific question about getting results does not mean it can't be part of the answer of the current, bit more broad question.Averyaveryl
The unhandled rejections in the accepted answer all come from the demonstrations of the "separate" style. The errors in the parts with Promise.all are all properly caught by the surrounding try/catch.Vasily
@Bergi, you are missing something: only one error could be caught with try / catch. The rest are unhandled.Averyaveryl
The other errors won't be available in the catch, but Promise.all still handles all rejections of the promises passed to it. You will not get an unhandled rejection event.Vasily
I'm sorry, but you are now starting to contradict yourself. 1. "The errors in the parts with Promise.all are all properly caught by the surrounding try/catch." 2. "The other errors won't be available in the catch,..."Averyaveryl
I used plural because there were multiple examples, not because there are multiple errors available in the catch block. Regardless, I hope you understood why I think this is not a useful answer and it doesn't need further explanation.Vasily
OK. Still, the other errors are unhandled. (nothing to do with "getting" unhandled rejection or not). I understand why you think this is no a useful answer, I hope others will find it very useful. People deserve better explanations and better examples.Averyaveryl
B
0

Resolves instead of Promises

const wait = (ms, data) => new Promise( resolve => setTimeout(resolve, ms, data) )
const reject = (ms, data) => new Promise( (r, reject) => setTimeout(reject, ms, data) )
const e = e => 'err:' + e
const l = l => (console.log(l), l)

;(async function parallel() {

  let task1 = reject(500, 'parallelTask1').catch(e).then(l)
  let task2 = wait(2500, 'parallelTask2').catch(e).then(l)
  let task3 = reject(1500, 'parallelTask3').catch(e).then(l)

  console.log('WAITING')

  ;[task1, task2, task3] = [await task1, await task2,  await task3]

  console.log('FINISHED', task1, task2, task3)

})()

As was pointed out in other answers, a rejected promise might raise an unhandled exception.
This one .catch(e => e) is a neat little trick that catches the error and passes it down the chain, allowing the promise to resolve, instead of rejecting.

If you find this ES6 code ugly see friendlier here.

Beadsman answered 17/2, 2019 at 3:46 Comment(2)
Why do only some of your lines start with a semicolon instead of ending all of them with a semicolon? Much more consistent and less confusing.Slob
@Slob I am anti-semicolonist and I only use them explicitly when the statement I am currently writing could interfere with a previous statement, such as console.log('WAITING') [task1]. In this case I will not return to the previous line, but instead I will put it at the start of the current statement where it belongs, e.g. in case I want to move the line somewhere else. See this video for reason why - it is good practice especially for beginners. There is also an eslint rule to enforce that.Beadsman

© 2022 - 2024 — McMap. All rights reserved.