What is the correct way to chain async calls in javascript?

Question

I'm trying to find the best way to create async calls when each call depends on the prior call to have completed. At the moment I'm chaining the methods by recursively calling a defined process function as illustrated below.

This is what I'm currently doing.

var syncProduct = (function() {
    var done, log;
    var IN_CAT = 1, IN_TITLES = 2, IN_BINS = 3;
    var state = IN_CAT;
    var processNext = function(data) {
        switch(state) {
            case IN_CAT:
                SVC.sendJsonRequest(url("/api/lineplan/categories"), processNext);
                state = IN_TITLES;
                break;
            case IN_TITLES:
                log((data ? data.length : "No") + " categories retrieved!");
                SVC.sendJsonRequest(url("/api/lineplan/titles"), processNext);
                state = IN_BINS;
                break;
            case IN_BINS:
                log((data ? data.length : "No") + " titles retrieved!");
                SVC.sendJsonRequest(url("/api/lineplan/bins"), processNext);
                state = IN_MAJOR;
                break;
            default:
                log((data ? data.length : "No") + " bins retrieved!");
                done();
                break;
        }
    }
    return {
        start: function(doneCB, logCB) {
            done = doneCB; log = logCB; state = IN_CAT;
            processNext();
        }
    }
})();

I would then call this as follows

var log = function(message) {
    // Impl removed.
}

syncProduct.start(function() {
    log("Product Sync Complete!");
}, log);

While this works perfectly fine for me I can't help but think there has to be a better (simpler) way. What happens later when my recursive calls get too deep?

NOTE: I am not using javascript in the browser but natively within the Titanium framework, this is akin to Javascript for Node.js.

Creekmore

Have you looked into "promises"? They may fit nice. This is a library implementing it for Node. – Cassiodorus

That library looks brilliant pimvdb, I think I'll be using this one. – Creekmore

Answer 1

There are lots of libraries and tools that do async chaining and control-flow for you and they mostly come in two main flavours:

1. Control-flow libraries

   For example, see async, seq and step (callback based) or Q and futures (promise based). The main advantage of these is that they are just plains JS libraries that ease the pain of async programming.

   In my personal experience, promise-based libraries tend to lead to code that looks more like usual synchronous code, since you return values using "return" and since promise values can be passed and stored around, similarly to real values.

   On the other hand, continuation-based code is more low level since it manipulates code paths explicitely. This can possibly allow for more flexible control flow and better integration with existing libraries, but it might also lead to more boilerplaty and less intuitive code.

2. Javascript CPS compilers

   Extending the language to add native support for coroutines/generators lets you write asynchronous code in a very straightforward manner and plays nice with the rest of the language meaning you can use Javascript if statements, loops etc instead of needing to replicate them with functions. This also means that its very easy to convert previously sync code into an async version. However, there is the obvious disadvantage that not every browser will run your Javascript extension so you will need to add a compilation step in your build proccess to convert your code to regular JS with callbacks in continuation-passing-style. Anyway, one promising alternative is the generators in the Ecmascript 6 spec - while only firefox supports them natively as of now, there are projects such as regenerator and Traceur to compile them back to callbacks. There are also other projects that create their own async syntax (since es6 generators hadn't come up back then). In this category, you will find things such as tamejs and Iced Coffeescript. Finally, if you use Node.js there you could also take a look at Fibers.

---

My recomendation:

If you just want something simple that won't complicate your build proccess, I would recomend going with whatever control-flow library best fits your personal style and the libraries you already use.

However, if you expect to write lots of complicated and deeply-integrated asynchronous code, I would strongly recommend at least looking into a compiler-based alternative.

Xenogamy

Excellent missingno, the Q library that pimvdb also mentioned in the comment to my question looks perfect. Could you give any feedback on libraries before I decide to go down a path? What I guess I don't want though is a completely blocking call on the first invoke. – Creekmore

@BrettRyan: Actually, I mostly work with Dojo toolkit stuff so I have most of my experience with their promise library and not much experience with the alternatives. All different styles should be able to do all the non-blocking stuff you need so you should look more for the programming style anyway. If I would have to start something right now I would heavily consider using a CPS compiler - I don't really know how it would actually turn out but right now I'm kind of tired of having to manually write CPS code and hoping for something better. – Xenogamy

Thanks missingno, I'm using the Appcelerator Titanium Mobile SDK so don't really want to put another tool into the tool chain. Experimenting with the Q library really does seem like a natural fit. – Creekmore

@TiborSzasz: You should ask that as a separate question instead of hiding it in these old comments. – Xenogamy

Yes, but it is not that interesting and lame. So could you make it work? – Peart

This is a Q&A site. How is asking a question "lame"? (And my examples were more for illustrative purposes. I don't know if all of them work with titanium or whatever other environment you want to use) – Xenogamy

I also have a slightly different take on the async chaining challenge: chainsjs.org - this is probably most similar to step, but has some interesting features that I haven't seen in other async libs – Discouragement