NodeJS local modules for complex application structures

Question

I'm currently part of team building a Windows 8 application using JavaScript. We are using npm and browserify to manage dependencies and convert our modules to AMD browser friendly format.

One issue we are running into is crazy require paths. This is because we have a top level folder inside our application "components". This folder contains a bunch of nested ui components/modules. These modules sometimes require lib utils and helpers, which reside in the lib directory.

So for example, a module living in "my/app/components/product/grid/item" might require a helper module which is located "my/app/lib/helpers/view".

The require path is a bit crazy and very ugly: require("../../../../lib/helpers/view");

We are doing a best to build in application in modular fashion. Now I would think the proper way to approach this is to have our components modules depend on these util helper modules. I could put the lib helpers into their own external private git repo, but that has been pain in terms of giving other teams access (plus git private repos are slow). Plus since those modules are only used in the application, it's a waste of time to make the change, push the changes, then go back to the application and npm update. This is fine for some, but if we really break this down, it could get old real fast.

I could do npm install "my/app/lib/helpers/view" inside the components package.json ? But npm install won't automatically do this for us.

I know of a few other ways around this (NODE_PATH, maybe use a npm install hook or maybe npm preinstall script), but wanted to know if anyone else had a similar problem and good solution.

Collinsia

My current approach is to run a batch script whenever we build that set the environment var for NODE_PATH. – Collinsia

Okay playing around a bit more npm link might be the way to go here. I make my application modules have package.json files, npm link inside those modules to make them available, then from the top level npm link them in. Only thing that sucks here is that I need to make sure we npm link on fresh install, as npm install won't link these modules for me. – Collinsia

I think separate repos is the way to go. – Disappointment

Especially for something like "helpers". – Disappointment

Basically whenever you end up going up one level, you're probably OK, but if you go up many levels and then back into another directory tree, you're looking at something that should be its own package. Whether you make it its own repo, or use a check-in-to-node_modules-technique like @substack recommends, is up to you. – Disappointment

Answer 1

You can put your "my/app/components/product/grid/item" files into node_modules/grid/item.js and then when you require('grid/item') in your application code you will get the file you want with a much terser require path syntax. Just check node_modules/grid/item.js and whichever other files into git. The node_modules/ directory need not even be at the top-level since the require algorithm used by node and browserify will search for node_modules/ directories from the current path all the way down to / until it finds a matching module.

Just make sure to add "grid" to the "bundledDependencies" array in your package.json so you don't accidentally install something over it.

You can read more about checking node modules into git.

Read the section of the browserify handbook about avoiding ../../../../../../ for more information.

NODE_PATH is always a bad idea and browserify doesn't support it. Don't ever use it ever.

Renaldo

Interesting. I knew NODE_PATH was the not the best approach, but I just recently watched TJ's video on modular web apps (tjholowaychuk.com/post/38571504626/…) with express and he mentions using NODE_PATH. – Collinsia

Yeah we are git ignore node_modules/*, but I thought shrinkwrap locks down ALL dependencies, even dependencies of your dependencies. That post seems to not mention that fact. – Collinsia

I rather not put these in top level, so like you mentioned, I would maybe create a directory structure where top level node_modules is for 3rd party, then inside my lib folder, there could be node_modules for application local modules. I guess my issue with this approach is that you would have to have a flat list of local modules. – Collinsia

Yes ok but substack in this case it seems Browserify won't transform raw source files like JSX or CoffeeScript. How to require a jsx file this way? – Optic

You say NODE_PATH is a bad idea. Can you please elaborate? – Grueling

Answer 2

One thing you could possibly do is create an alias for your helpers in your require config...

require.config({
    paths: {
        "helpers": "my/app/lib/helpers"    
    }
});

That would cut down on some of your long paths.

Phenylamine

Yeah I think this is only fore requirejs, which we are not using. See above, we are using browserify to wrap our modules. In addition, I'd like this solution to work on the server too. – Collinsia

Answer 3

The problem of the require() function is that the paths are relative from the current file. You could put your modules inside the node_modules directory but this is the worst thing you could do. node_modules is the directory where live all the third-party modules. If you follow this simple rule it's very easy and handy to stay always up to date, you can remove all the dependencies (removing the node_modules) and just doing npm install.

The best solution is to define your own require function and make it global. For example:

Your project structure is:

my-project
| tools
|- docs
|- logs
|- conf
`- src
   |- node_modules
   |- package.json
   |- mod.js
   |- a
   |  `- b
   |     `- c.js
   `- d
      `- app.js

mod.js

global.mod = function (file){
  return require ("./" + file);
};

app.js

//This should be the first line in your main script
require ("../mod");

//Now all the modules are relative from the `src` directory
//You want to use the a/b/c.js module
var c = mod ("a/b/c");

That's all, easy. If you want to get a third-party module located in node_modules use require(). If you want to get your own modules use mod().

And remember, node_modules is only for third-party modules, rule nº1.

Sosthina

A custom require function like you have here with require("./" + file) is not statically analyzable and won't work with browserify. Putting your own modules in node_modules/ is perfectly acceptable and is the reason why there is such a thing as "bundledDependencies" in the first place. – Renaldo

If it's not compatible with browserify then make it compatible with browserify. My solution it's more cleaner than putting your stuff inside node_modules – Sosthina