Design pattern for managing multiple asynchronous JavaScript operations

Question

I'm looking for a good design pattern for keeping track of a bunch of different asynchronous JavaScript activities (images loading, multiple AJAX calls, sequenced AJAX calls, etc…) that's better than just a lot of custom callbacks and custom state variables. What would you suggest I use? Is there any type of queue system with the ability to have logic beyond just sequencing?

Example Problem

I have a startup sequence that involves a number of asynchronous processes (loading images, waiting for timers, making some ajax calls, doing some initialization). Some of the asynch processes can be launched to run at the same time (loading of images, AJAX calls) and some have to be sequenced (run AJAX call #1, then AJAX call #2). Right now, I've got everything running off callback functions and a bunch of global state that keeps track of what has or hasn't completed. It works, but it's quite messy and I've had a few bugs because of the complication of making sure you handle all the sequencing possibilities right and error conditions.

When you have problems, it's also quite a pain to debug because it's like the Heisenberg uncertainty principle. As soon as you set a breakpoint anywhere in the sequence, everything changes. You have to put all sorts of debugging statements in the code to try to discern what's happening.

Here's a more specific description:

1. There are three images loading. As soon as one specific image is loaded, I want to display it. Once it's been displayed for a certain amount of time, I want to display the second image. The third one goes in a queue for later display.

2. There are three AJAX calls that must happen in consecutive order (the output of one is used as part of the input of the next).

3. When the AJAX calls are done, there's a bunch of JS processing of the results to do and then two more images need to get loaded.

4. When those two images are loaded, there's some more display stuff to do.

5. When all that is done, you examine how long one of the images has been displayed and if enough time has passed, display the next image. If not, wait some more time before displaying the next image.

Each step has both success and error handlers. Some of the error handlers kick off alternate code that can still continue successfully.

I don't expect anyone to follow the exact process here, but just to give folks an idea of the type of logic between the steps.

Edit: I came across YUI's AsyncQueue which isn't a complete solution for the type of problem I have, but is in the same space. It seems to be more for sequencing or ordering a bunch of async operations, but I don't see how it helps with the type of decision making I have.

Honky

did you take a look at Lab.js, i think it should be able to help you – Incongruity

My first read of lab.js is that it's for loading multiple scripts as fast as possible in a controlled sequence. Is it somehow suitable for sequencing a variety of types of asynch operations all from the same script? – Honky

Would love to see some more responses to this question. There must be more out there as it's such a common issue now. – Honky

Answer 1

With promises now standard in ES6 and many good promise libraries extending that for both new features and backwards compatibility, it seems that promises are the way to go here.

The solution starts with taking each async operation and creating a wrapper that returns a promise:

For loading an image:

function loadImage(url) {
    return new Promise(function(resolve, reject) {
        var img = new Image();
        img.onload = function() {
            resolve(img);
        };
        img.onerror = img.onabort = function() {
            reject(url);
        };
        img.src = url;
    });
}

For making an Ajax call (simplified version):

function ajaxGet(url) {
    return new Promise(function(resolve, reject) {
        var req = new XMLHttpRequest();
        req.addEventListener("load", resolve);
        req.addEventListener("error", reject);
        req.addEventListener("abort", reject);
        req.open("GET", url);
        req.send();
    });
}

For waiting a particular amount of time before executing an operation, you can even making a promise version of setTimeout() so it can be chained with other promise operations:

// delay, return a promise
// val is optional
function delay(t, val) {
    return new Promise(function(resolve) {
        setTimeout(function() {
            resolve(val);
        }, t);
    });
}

Now, these can be combined to create the logic the question asked for:

There are three images loading. As soon as one specific image is loaded, I want to display it. Once it's been displayed for a certain amount of time, I want to display the second image. The third one goes in a queue for later display.

// start all three images loading in parallel, get a promise for each one
var imagePromises = [url1, url2, url3].map(function(item) {
    return loadImage(item);
});

// display the three images in sequence with a delay between them
imagePromises.reduce(function(p, item) {
    return p.then(function() {
        // when the next image is ready display it
        return item.then(function(img) {
            displayImage(img);
            return delay(15 * 1000);
        });
    });
}, Promise.resolve());

This use of .reduce() shows a classic design pattern for sequencing a series of operations on an array using promises.

There are three AJAX calls that must happen in consecutive order (the output of one is used as part of the input of the next).

and

When the AJAX calls are done, there's a bunch of JS processing of the results to do and then two more images need to get loaded.

var p = ajaxGet(url1).then(function(results1) {
    // do something with results1
    return ajaxGet(url2);
}).then(function(results2) {
    // do something with results2
    return ajaxGet(url3); 
}).then(function(results3) {
    // process final results3 here
    // now load two images and when they are loaded do some display stuff
    return Promise.all(loadImage(imgx), loadImage(imgy)).then(function(imgs) {
        doSomeDisplayStuff(imgs);
    });
});

Honky

Answer 2

Take a look at the concept of Promises/A. jQuery implements this with the jQuery.Deferred object.

Here is a nice article showing how it could be useful for your situation. I asked a similar question a while back.

Mcallister

Answer 3

Though Promises afford us cleaner code we can do better with Generators. I wrote up a post on how to use Generators in Tame Async JavaScript with ES6. Using the pattern described will make handling complex async interactions easier to reason about and sets up a mental model for await capabilities planned for ES7.

Marzi

Answers where all the the "meat" of the answer is in an outside resource and not in the actual answer are not considered appropriate here. If you can include the meat of your article in your actual answer, it would make it a better and more meaningful answer. FYI, I had trouble following your article. I know promises really well, but don't know generators at all. If you can explain it in more detail here in your answer, that would be great. – Honky