Koa / Co / Bluebird or Q / Generators / Promises / Thunks interplay? (Node.js) [closed]

Question

I'm investigating building a web app in part with Koa, but I don't quite have a handle on the hows, whens, and whys of choosing between - and applying - the range of supportive "making async easier" technologies/approaches (listed below).

Overall the disparate guidance on the web about this subject still leaves things blurry, especially in respect to evolving best practices, or at least better ones, and under what scenarios. There seems to be little or nothing on the web that puts it all in context.

I'm hoping the responses to this big arse sprawling post can correct that. Also maybe the questions below can inspire someone to write a thorough blog post or the like to address this matter. My sense is I'm not even close to the only one who would benefit from that.

So I'd be pleased if the bright community could help answer and provide clarity to the following questions in respect to the technologies listed below (in bold type):

-- a) How, and under what circumstance (as applicable) are they complements, supplements, substitutes, and/or overlapping solutions to one another?

-- b) What are their trade-offs in respect to speed-performance, error handling ease, and debugging ease?

-- c) When, where, and why may it be better to use "this" versus "that" technology, technologies-combo, and/or approach?

-- d) Which technologies or approaches, if any, may be "dimming stars".

(Hoping that the opinions that are part of answers can be well explained.)

==============================

Technologies:

* Koa *

My understanding:

Koa is a minimal foundation for build Node apps geared for taking advantage of ECMAScript-6 features, one feature in particular being generators.

* Co *

My understanding:

-- Co is a library of utilites for running ECMAScript-6 generators (which are native to Node .011 harmony), with the goal to allieve some/much(?) of the need to write boilerplate code for running and managing generators.

-- Co is intrinsically part of Koa(?).

Specific questions:

-- If and how does one use Co differently in Koa than in a non-Koa context. In other words, does Koa wholly facade Co?

-- Could Co be replaced in Koa with some other like generator library if there is/was a better one? Are there any?

* Promise Libraries such as "Q" and Bluebird *

My understanding:

-- They are in a sense "polyfills" for implmententing the Promises/A+ spec, if and until Node natively runs that spec.
-- They have some further non-spec convenience utilities for facilitating the use promises, such as Bluebird's promisfyAll utility.

Specific questions:

-- My understanding is the ECMAScript-6 spec does/will largely reflect the Promises/A+ spec, but even so, Node 0.11v harmony does not natively implement Promises. (Is this correct?) However when it does, will technologies such as Q and Bluebird be on their way out?

-- I've read something to the effect that "Q" and Bluebird support generators. What does this mean? Does it mean in part that, for example, they to some degree provided the same utility as Co, and if so to what degree?

* Thunks and Promises *

I think I have an fair handle on what they are, but hoping someone can provide a succinct and clear "elevator pitch" definition on what each is, and of course, as asked above, to explain when to use one versus the other -- in a Koa context and not in it.

Specific questions:

-- Pro and cons to using something like Bluebird's promisfy, versus say using Thunkify (github com/visionmedia/node-thunkify)?

==============================

To give some further context to this post and its questions, it might be interesting if Koa techniques presented in the following webpages could be discussed and contrasted (especiallly on a pros vs cons basis):

-- a) www.marcusoft . net/2014/03/koaintro.html (Where's the thunks or promises, or am I not seeing something?)

-- b) strongloop . com/strongblog/node-js-express-introduction-koa-js-zone (Again, where's the thunks or promises?)

-- c) github . com/koajs/koa/blob/master/docs/guide.md (What does the "next" argument equate to, and what set it and where?)

-- d) blog.peterdecroos . com/blog/2014/01/22/javascript-generators-first-impressions (Not in a Koa context, but presents the use of Co with a promise library (Bluebird), so I'm assuming the technique/pattern presented here lends itself to usage in Koa(?). If so, then how well?

Thanks all!

Sining

Bluebird promisify with Bluebird coroutine for generator is the fastest method by far. It also happens to offer the best debuggability options with long stack traces at the moment. I'd say it's a good bet. – Sforza

Also, this is like a million questions. Some good, some too opinion based. – Sforza

Hi Ben - Yeah I know it's ventures into opinion, but when asking for considerations for put a system, o – Sining

Thanks Ben. (mis submit above) Read good things about Bluebird. It seems though that bb-coroutine doesn't work w/ thunks, only promises. So what's the downside to losing the option to work w/ thunks, particularly in the context of Koa? (Note "https://mcmap.net/q/330075/-co-js-and-bluebird-js-what-39-s-the-difference" alludes to BB working someday w/ thunks.) 2nd, would bb-coroutine completely replace Co in Koa? If so, how? Seems the two are joined at the hips? Regarding opinion, "yeah", but when the topic is considerations for building a system, opinion becomes part of the equation. – Sining

Bluebird works with thunks, take a look at addYieldHandler. Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise. – Sforza

Co is a library created by express author to write non-blocking code easily. Usually we have to work with callbacks and async functions, which can get ugly. Chaining them elegantly can be a challenge. In express, server chains various middleware. Koa is like express, it's a middleware. But it uses co for managing the internals. The catch is that generators are good but they are synchronous. So you need promises to work with them. Co does this for you. – Marciamarciano

This isn't a QA but a discussion. Therefore it's suited for a forum, not for SO. – Croupier

I appreciate the effort you put into this so called question, unlike some people, but it's off topic here. Try the chat rooms. – Deadening

@Connor chat rooms won't leave the information available to help future viewers – Outpour

Answer 1

I've been working almost extensively with generators for a month now so maybe I can take a stab at this. I'll try to keep the opinions to a minimum. Hopefully it helps clarify some of the confusion.

Part of the reason for the lack of best practices and better explanations is that the feature is still so new in javascript. There are still very few places that you can use generators node.js and firefox being the most prominent, though firefox deviates from the standard a bit.

I would like to note that there are tools like traceur and regenerator that will let you use them for development and allow you to turn them into semi-equivalent ES5 so if you find working with them enjoyable then there's no reason not to start using them unless you're targeting archaic browsers.

Generators

Generators weren't originally thought of as a way to handle asynchronous control flows but they work wonderfully at it. Generators are essentially iterator functions that allow their execution to be paused and resumed through the use of yield.

The yield keyword essentially says return this value for this iteration and I'll pick up where I left off when you call next() on me again.

Generator functions are special functions in that they don't execute the first time they're call but instead return an iterator object with a few methods on it and the ability to be used in for-of loops and array comprehensions.

send(),: This sends a value into the generator treating it as the last value of yield and continues the next iteration

next(),: This continues the next iteration of the generator

throw(): This throws an exception INTO the generator causing the generator to throw the exception as though it came from the last yield statement.

close(): This forces the generator to return execution and calls any finally code of the generator which allows final error handling to be triggered if needed.

Their ability to be paused and resumed is what makes them so powerful at managing flow control.

Co

Co was built around the ability of generators to make handling flow control easier. It doesn't support all of the things that you can do with generators but you can use most of them through it's usage with less boilerplate and headache. And for flow control purposes I haven't found that I needed anything outside of what co provides already. Although to be fair I haven't tried sending a value into a generator during flow control but that does bring up some interesting possibilities....

There are other generator libraries out there some of them that I can think of off the top of my head are suspend, and gen-run. I've tried them all and co offers the most flexibility. Suspend may be a little easier to follow if you're not accustomed to generators yet but I can't say that with authority.

As far as node and best practices go I'd say co is currently winning hands down with the amount of support tools that have been created to go with it. With suspend the most likely runner up.

Co works with both promises and thunks and they are used for yield statement so that co knows when to continue execution of the generator instead of you manually having to call next(). Co also supports the use of generators, generator functions, objects and arrays for further flow control support.

By yielding an array or an object you can have co perform parallel operations on all of the yielded items. By yielding to a generator or generator function co will delegate further calls to the new generator until it is completed and then resume calling next on the current generator, allowing you to effectively create very interesting flow control mechanisms with minimal boilerplate code.

Promises

While I said I'd keep opinions to a minimum I would like to state that to me promises are probably the hardest concept to grasp. They are a powerful tool for maintaining code but they are hard to grasp the inner workings of and can come with quite a few gotchas if used for advanced flow control.

The easiest way that I can think of to explain promises is that they are an object returned by a function that maintains the state of the function and a list of callbacks to call when the a specific state of the object is or has been entered into.

The promise libraries themselves won't be going anywhere anytime soon. They add a great deal of nice to haves for promises included done() which didn't make it into the ES6 spec. Not to mention the fact that the same libraries can be used on the browser and in node we'll have them for a good long while.

Thunks

Thunks are just functions that take a single parameter callback and return another function that they are wrapping.

This creates a closure that allows the calling code to instantiate the function passing in its callback so that it can be told when the method is complete.

Thunks are fairly straight forward to understand and use in my opinion but they aren't the right tool for everything. For example spawn is a major pain to create a thunk for, you can do it but it's not easy.

Thunks vs. Promises

These aren't mutually exclusive and can easily be used together, but it's usually better for your sanity to pick one and stick with it. Or at the very least pick a convention so you can easily tell which is which. Thunks run faster from my experience but I haven't benchmarked it. Most of this is probably because it's a smaller abstraction and doesn't have error handling mechanisms built in.

You'll usually be building something that requires error handling though so the overall performance gains of thunks could easily even out or side in the favor of promises depending on your code.

When to Use

Generators - When you can safely say that your application will be able to run on the bleeding edge, whether it's firefox only for the browser or node > 0.11.3

I've been using them extensively at the company I'm out now and couldn't be happier with the control flow mechanisms and lazy evaluation that they allow.

Promises vs. Thunks - This is really up to you and how comfortable you are working with each. They don't provide the same benefits nor do they solve the same problem. Promises help deal with the async problem directly, thunks just ensure a function takes the needed callback parameter for other code to pass in.

You can use them both together and as long as you can keep it so that it's obvious which is which you won't have a problem.

Promises/Thunks with Generators - I suggest doing this anytime you are using generators for control flow. It's not necessary but it's easier just like using co as an abstraction for control flow with generators is easier. Less code to type, easier maintenance, and less possibilities that you'll hit an edge case that somebody else hasn't run into yet.

Koa

I'm not going to go into a lot of detail on koa. Suffice it to say that is similar to express but written to take advantage of generators. This does give it some unique advantages such as easier error handling and cascading middleware. There were ways to accomplish all of these tasks before but they weren't elegant and sometimes not the most performant.

Special Note: Generators open up a door of possibilities that we really haven't explored yet. Just like they can be used for control flow when that wasn't their initial design I'm positive they can be used to solve a lot of other problems that we normally have problems with in javascript. It will probably be brighter minds than me that find out how else we can use them but I'd at least start playing around with them and getting a better understanding of what they're capable of. There's still more goodies for generators coming in ES.next.

Linalool

I'm sorry, this answer is not bad and it seems like you've spent a lot of time on it. While I value the contribution as it stands I do not find it useful in the context. It is way too specific for the way too specific question OP asked and the 'when to use' section is very opinion based. – Sforza

The promises section is also very lacking, but that's besides the point. Also, generators have been used for async flow control in other languages for many years now.. C# even has them built in via the async/await construct which is also a worked-on proposal for ES7. – Sforza

Agreed. It's kind of hard to keep answers unopinionated when it comes to something that's still relatively new and unexplored.Hopefully it will at least help alleviate some of the confusion on the what even if not as much on the when and why. – Linalool

While other languages have had generators for a long time now, not just C# but also python, the javascript domain itself has a lot of oddities particularly on the browser side where their use hasn't been explored yet. Just like any other language construct they're no panacea but they do certainly make life easier :). – Linalool

Thot, thanks for the response. .... Ben, likewise for your responses. I'm realizing that I'm unclear on what is the distinction between coroutines versus generators. My sense was that the former is simply the control logic for running the latter, and advancing past yields. Maybe I am right on this, at least in part, but I now sense the concept of coroutines distinct and standalone from generators(?). – Sining

Ben (or anyone) - wondering if you could have a go at my question in #23037008. The question relates to this post, which about comparing and contrasting varying approaches to structuring async in Koa. (Note I asked the question in a "Answer", since I don't yet have the reputation put it in a "Comment". Thanks. – Sining