JavaScript variables hoisting in nodejs/async

Question

In the next example I don't have access to variable "locals" inside the functions "fetcher", "parser" and "saveToDb".

var parser = require('parser.js');
var fetcher = require('fetcher.js');
var saveToDb = require('models/model.js');
var async = require('async');


function task() {
    var locals = []    //<-- declared here
    async.series([
        fetcher,    //<--  can not access "locals"
        parser,     //<--  can not access "locals"
        saveToDb    //<--  can not access "locals"
    ],
            function (err) {
                if (err) return callback(err);
                callback(null);
    });
}

In the next example "local"s is accessible. I just copyed the functions declarations from the requested modules, and pasted them straight inside "async.series".

var async = require('async');

function task() {
    var locals = []    //<-- declared here
    async.series([
        function(callback) {// <-- can access "locals"},  
        function(callback) {// <-- can access "locals"},
        function(callback) {// <-- can access "locals"}
    ],
            function (err) {
                if (err) return callback(err);
                callback(null);
    });
}

While this works - I do want to keep my code modular. How can I fix that ? Or - what I forgot here about the fundamentals of JavaScript ?

Thanks.

Britton

It could be said that you are forgetting one thing: That your code won't be very modular if it could access more or less random variables (in this case the locals variable) defined elsewhere. Making something modular usually takes more than just spreading things around files, it's also about making relationships between different parts of your code explicit, for example by passing all data a function needs as arguments. – Mae

Answer 1

In the first example, the callbacks live in another scope so can't access locals.

You could create partial functions that get the locals variable passed as first argument, but that would require you to rewrite your callbacks.

// creating a partial
async.series([
  fetcher.bind(fetcher, locals),
  parser.bind(parser, locals),
  saveToDb.bind(saveToDb, locals)
], ...)

// new function signatures
function fetcher (locals, callback) { ... }
function parser  (locals, callback) { ... }
function saveToDb(locals, callback) { ... }

Billbillabong

Why the bind? I guess that because I don't use "this" inside my specific callback - it does not really matter - but I do want to understand why you added the bind. Thanks robert. – Britton

bind serves two purposes: determine what this is in the function, but also to create partial functions which have pre-specified arguments 'attached' to them (see the MDN explanation, and here a simple gist to demonstrate the principle). – Billbillabong

I think I got this part right now, but will that work even if "async.series" takes no arguments except "callback" ? – Britton

Sure :) I updated my gist to show how you can use it with async. – Billbillabong

I think the fetcher.bind(fetcher, locals) could better be fetcher.bind(undefined, locals) Specifying the original function as value for this looks weird to me. – Mae

@MerynStol sure, or null or whatever :) I have the habit of passing the original object to bind usually, although in this case it doesn't make much sense I have to admit (edit: okay, here's a convoluted gist to show what you might be missing ;) – Billbillabong

@Billbillabong Seems I'm not missing out on much, thankfully. ;) – Mae

Robert - it seems to almost work. Every function now has access to the value of "locals" - but only to the first value - it's not "two way binding" - am I wrong ? – Britton

off-course, the explicit arguments produce a new inner scope. I can't change "locals" across functions this way... – Britton

@Britton it sounds like you want to use locals to share/pass some state between the different callbacks. If so, you should consider using async.waterfall instead of async.series. – Billbillabong

(although this gist seems to suggest that you could pass data around with the method I proposed) – Billbillabong

@robertklep, yeah just saw that and will use that. but I still wonder for my understanding - if there's a way without waterfall, maybe using a closure and self invoked functions... – Britton

@robertklep, I saw the gist, there's no write access to the outer locals from inside the functions when the arguments are passed explicitly(I mean it's read only)... as far as I can see and checked again. – Britton

@Britton oh, that's absolutely right, it's not a magic trick to break out of the current scope :) – Billbillabong

Answer 2

You can have the 3 mentionned functions defined like this :

function fetcher(locals) {
  return function _fetcher(callback) {
    // this function body will have access to locals
  }
}

and rewrite your code as:

function task() {
    var locals = []    //<-- declared here
    async.series([
        fetcher(locals),    //<--  can now access "locals"
        parser(locals),     //<--  can now access "locals"
        saveToDb(locals)    //<--  can now access "locals"
    ],
            function (err) {
                if (err) return callback(err);
                callback(null);
    });
}

Section

I'm not sure if "async.series" takes arguments except "callback", I'll check that. Thanks matehat. – Britton

Answer 3

IF i understand your problem correctly this is a javascript 101 question.

fetcher can be a factory that creates a concrete fetcher.

var fetcher = function(locals){
 locals=locals||[] // optional , assign a default , whatever ...
 return function(){
   //do some operation with locals.
   doSomething(locals);
 }
}

then in your script

 async.series([
        fetcher(locals),    //<--  will return the callback to use
        parser(locals),     //<--  will return the callback to use
        saveToDb(locals)    //<--  will return the callback to use
    ],

i believe it is a kind of currying.

Concrete

"async.series" takes no arguments except "callback" (I think), and also "locals" is my main "data" - I cannot delete it each time. Thanks mpm. – Britton