Natively import ES module dependencies from npm without bundling/transpiling first-party source

Question

Background

I'm trying to create a "buildless" JavaScript app, one where I don't need a watch task running to transpile JSX, re-bundle code, etc every time I save any source file.

It works fine with just first-party code, but I'm stuck when I try to import dependencies from npm.

Goal

I want to achieve this kind of workflow:

1. npm install foo (assume it's an ES module, not CommonJS)
2. Edit source/index.js and add import { bar } from 'foo'
3. npm run build. Something (webpack, rollup, a custom script, whatever) runs, and bundles foo and its dependencies into ./build/vendor.js (without anything from source/).
4. Edit index.html to add <script src="build/vendor.js" type="module"...
5. I can reload source/index.js in my browser, and bar will be available. I won't have to run npm run build until the next time I add/remove a dependency.

I've gotten webpack to split dependencies into a separate file, but to import from that file in a buildless context, I'd have to import { bar } from './build/vendor.js. At that point webpack will no longer bundle bar, since it's not a relative import.

I've also tried Snowpack, which is closer to what I want conceptually, but I still couldn't configure it to achieve the above workflow.

I could just write a simple script to copy files from node_modules to build/, but I'd like to use a bundled in order to get tree shaking, etc. It's hard to find something that supports this workflow, though.

Killing

The logical question is why are you using webpack at all if you don't want to have "build" your project. If you use webpack, you will have to build. That's how it works. I've built dozens of apps and none of them use webpack or any "packaging" tool. If you choose to use webpack for the features that it provides, then you are choosing to have to build your app. – Vigil

It's a compromise. Ideally it wouldn't be needed at all, but it seems like it's needed for production, just not during the dev workflow. If I can do it all without webpack, all the better. How do you handle dependencies in your apps? – Killing

Er, to clarify, there's two reasons: 1) In the dev workflow I'm using a modern browser, but for production I need to support IE10+, so I need to transpile to ES5, etc. I also want to transpile away HTM, etc for performance. 2) Dependencies. Say I want to import eff-diceware-passphrase, it only provides a CommonJS module, and has its own dependencies. The build step seems necessary even just to import that in the dev workflow. – Killing

Well, if you're transpiling, you're always going to be building. If I was transpiling for production, I'd usually be transpiling in my dev environment too so I'm testing/running the same code that will be run in production. It seems you could build your external modules and their dependencies each into their own separately imported bundle so as long as they don't get updated, you don't have to rebuild them. Probably not super efficient for production because some code might be duplicated if you don't let the bundler analyze everything together, but it could be fine for the dev environment. – Vigil

For example, you could build eff-diceware-passphrase and it's dependencies into one bundled script that you import and that build would be a one-time thing until you update to a newer version of that module. It's analogous to building a DLL once in C++ and not rebuilding it every time you do a new build if nothing changed in it. – Vigil

Yeah, I think that's basically what I was proposing in the question, except I only want to run it during build, not watch, because of the devex tradeoffs. YMMV, but that's acceptable to me. I do want all deps to be in a single "vendor" build, and for the tool to do tree shaking. I just haven't been able to get it to work, partially because of the circular problem w/ bare module specifiers vs path specifiers, and partially because webpack config feels like voodoo to me most of the time. – Killing

Answer 1

I figured out how to do this, using Import Maps and Snowpack.

High-Level Explanation

I used Import Maps to translate bare module specifiers like import { v4 } from 'uuid' into a URL. They're currently just a drafted standard, but are supported in Chrome behind an experimental flag, and have a shim.

With that, you can use bare import statements in your code, so that a bundler understands them and can work correctly, do tree-shaking, etc. When the browser parses the import, though, it'll see it as import { v4 } from 'http://example.org/vendor/uuid.js', and download it like a normal ES module.

Once those are setup, you can use any bundler to install the packages, but it needs to be configured to build individual bundles, instead of combining all packages into one. Snowpack does a really good job at this, because it's designed for an unbundled development workflow. It uses esbuild under the hood, which is 10x faster than Webpack, because it avoids unnecessarily re-building packages that haven't changed. It still does tree-shaking, etc.

Implementation - Minimal Example

index.html

<!doctype html>
<!-- either use "defer" or load this polyfill after the scripts below-->
<script defer src="es-module-shims.js"></script>
<script type="importmap-shim">
{
  "imports": {
    "uuid": "https://example.org/build/uuid.js"
  }
}
</script>

<script type="module-shim">
  import { v4 } from "uuid";

  console.log(v4);
</script>

snowpack.config.js

module.exports = {
    packageOptions: {
        source: 'remote',
    },
};

packageOptions.source = remote tells Snowpack to handle dependencies itself, rather than expecting npm to do it. Run npx snowpack add {module slug - e.g., 'uuid'} to register a dependency in the snowpack.deps.json file, and install it in the build folder.

package.json

"scripts": {
    "build":  "snowpack build"
}

Call this script whenever you add/remove/update dependencies. There's no need for a watch script.

Implementation - Full Example

Check out iandunn/no-build-tools-no-problems/f1bb3052. Here's direct links to the the relevant lines:

• snowpack.config.js
• snowpack.deps.json
• package.json
• core.php outputs the shim
• plugin.php - outputs the import map
• passphrase-generator.js - imports the modules. (They're commented out in this example, for reasons outside the scope of this answer, just uncomment them, run the bundle script, and they'll work).

Killing

Answer 2

If you are willing to use an online service, the Skypack CDN seems to work nicely for this. For instance I wanted to use the sample-player NPM module and I've chosen to use a bundle-less workflow for my project using only ES6 modules as I'm targeting embedded Chromium latest version so don't need to worry about legacy browser support, so all I needed to do was:

import SamplePlayer from "https://cdn.skypack.dev/sample-player@^0.5.5";

// init() once the page has finished loading.
window.onload = init;

function init() {
  console.log('hello sampler', SamplePlayer)
}

and in my html:

  <script src="./src/sampler/sampler.js" type="module"></script>

And of course you could just look inside the JS file the CDN generates at the above url and download the generated all-in-one js file it points to, in order to use it offline as well if needed.

Excursion

That's definitely a good approach for some circumstances. IIRC though, Skypack sometimes serves different code to different browsers (for polyfilling, etc), so I'm not sure it'd be reliable to download one copy and serve it to everyone. – Killing