How to run Generator Functions in Parallel?

Question

Assuming I have a Koa web server with an endpoint like this:

const perform = require(...); // some generator function

exports.endpoint = function* () {

    var results = yield getResults();

    // Respond the results
    this.body = results;
}

exports.getResults = function* () {

    var actions = [...];
    var results = [];

    for (var action of actions) {

        var result = yield perform(action);

        results.push(results);
    }

    return results;
}

Now the client will get the respond after ALL the actions are performed obviously. but the things is each action is dependent on the completion of the previous.

Is there a way to execute them in parallel?

Note: Turning them to Promises is not an option, unless I can somehow return the results, and not resolve() them.

Thetisa

Answer 1

co turns the generator function to Promises, and executes them async. Promise.all waits for all of them to finish:

exports.getResults = function* () {

    var actions = [...];

    return yield Promise.all(actions.map(function(action) { 
        return co(function*() { 
            return yield perform(action); 
        } 
    }));
}

Thetisa

Answer 2

If the generators are used as coroutines, by simulating the async/await flow, then you should be able to do:

var results = yield Promise.all(actions.map(action => perform(action)));

or even:

var results = yield Promise.all(actions.map(perform));

I'm not sure about the exact usage here but when you use generators with co or Bluebird.coroutine then you're already using promises, so you may as well use them more explicitly.

So, instead of:

exports.getResults = function* () {

    var actions = [...];
    var results = [];

    for (var action of actions) {

        var result = yield perform(action);

        results.push(results);
    }

    return results;
}

you can try:

exports.getResults = function* () {

    var actions = [...];

    return yield Promise.all(actions.map(perform));
}

Lecroy

Problem is perform(action) returns suspended Generator, and will not execute the actual perform(). When yielded then Koa (or Bluebird.coroutine I believe) actually executes the function. – Thetisa

In order to actually run the perform function I can use co lib, does this solution make sense? return yield Promise.all(actions.map(function(action) { return co(function*() { return yield perform(action); } })); – Thetisa

@ShaiBen-Dor Yes, it makes sense, co is very good for things like that. You may also be able to use co.wrap (part of the co module) and coroutine from Bluebird - see bluebirdjs.com/docs/api/promise.coroutine.html – Lecroy

Answer 3

if you are using redux saga than follow this link

https://redux-saga.js.org/docs/advanced/RunningTasksInParallel/

Governorship