Compile a package that depends on ESM only library into a CommonJS package

Question

I am working on a package that depends on a ESM only library: unified and I exposed my npm package as CommonJS library.

When I called my package in an application, node gives me this error message:

require() of ES Module node_modules\unified\index.js not supported

The error message is obvious since we are not allowed to require a ESM module, but didn't I already tell Typescript to compile the source code into CommonJS format?

---

References:

1. ESM vs CommonJS
2. How to Create a Hybrid NPM Module for ESM and CommonJS

Protease

Answer 1

Summary

You can't use static import statements in CJS: there's no way around it.

However it is possible to use ES modules via dynamic import statements if you only need to use the module in async contexts. However, the current state of TypeScript introduces some complexity in regard to this approach.

---

How-to

Consider this example in which I've setup a CJS TS repo using the module you mentioned, and I've configured the npm test script to compile and run the output. I've put the following files into an empty directory (which I've named so-70545129 after the ID of this Stack Overflow question):

Files

./package.json

{
  "name": "so-70545129",
  "version": "1.0.0",
  "description": "",
  "type": "commonjs",
  "main": "dist/index.js",
  "scripts": {
    "compile": "tsc",
    "test": "npm run compile && node dist/index.js"
  },
  "author": "",
  "license": "MIT",
  "devDependencies": {
    "@types/node": "^17.0.5",
    "typescript": "^4.5.4"
  },
  "dependencies": {
    "unified": "^10.1.1"
  }
}

./tsconfig.json

{
  "compilerOptions": {
    "exactOptionalPropertyTypes": true,
    "isolatedModules": true,
    "lib": [
      "ESNext"
    ],
    "module": "CommonJS",
    "moduleResolution": "Node",
    "noUncheckedIndexedAccess": true,
    "outDir": "dist",
    "strict": true,
    "target": "ESNext",
  },
  "include": [
    "./src/**/*"
  ]
}

./src/index.ts

import {unified} from 'unified';

function logUnified (): void {
  console.log('This is unified:', unified);
}

logUnified();

---

Now, run npm install and run the test script:

$ npm install
--- snip ---

$ npm run test

> [email protected] test
> npm run compile && node dist/index.js


> [email protected] compile
> tsc

/so-70545129/dist/index.js:3
const unified_1 = require("unified");
                  ^

Error [ERR_REQUIRE_ESM]: require() of ES Module /so-70545129/node_modules/unified/index.js from /so-70545129/dist/index.js not supported.
Instead change the require of /so-70545129/node_modules/unified/index.js in /so-70545129/dist/index.js to a dynamic import() which is available in all CommonJS modules.
    at Object.<anonymous> (/so-70545129/dist/index.js:3:19) {
  code: 'ERR_REQUIRE_ESM'
}

For reference, here's the output: ./dist/index.js:

"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
const unified_1 = require("unified");
function logUnified() {
    console.log('This is unified:', unified_1.unified);
}
logUnified();

The error above explains the problem (which I summarized at the top of this answer). TypeScript has transformed the static import statement into an invocation of require because the module type is "CommonJS". Let's modify ./src/index.ts to use dynamic import:

import {type Processor} from 'unified';

/**
 * `unified` does not export the type of its main function,
 * but you can easily recreate it:
 *
 * Ref: https://github.com/unifiedjs/unified/blob/10.1.1/index.d.ts#L863
 */
type Unified = () => Processor;

/**
 * AFAIK, all envs which support Node cache modules,
 * but, just in case, you can memoize it:
 */
let unified: Unified | undefined;
async function getUnified (): Promise<Unified> {
  if (typeof unified !== 'undefined') return unified;
  const mod = await import('unified');
  ({unified} = mod);
  return unified;
}

async function logUnified (): Promise<void> {
  const unified = await getUnified();
  console.log('This is unified:', unified);
}

logUnified();

Run the test script again:

$ npm run test

> [email protected] test
> npm run compile && node dist/index.js


> [email protected] compile
> tsc

node:internal/process/promises:246
          triggerUncaughtException(err, true /* fromPromise */);
          ^

Error [ERR_REQUIRE_ESM]: require() of ES Module /so-70545129/node_modules/unified/index.js from /so-70545129/dist/index.js not supported.
Instead change the require of /so-70545129/node_modules/unified/index.js in /so-70545129/dist/index.js to a dynamic import() which is available in all CommonJS modules.
    at /so-70545129/dist/index.js:11:52
    at async getUnified (/so-70545129/dist/index.js:11:17)
    at async logUnified (/so-70545129/dist/index.js:16:21) {
  code: 'ERR_REQUIRE_ESM'
}

Roadblock

Hmm 🤔, didn't we just fix this?? Let's take a look at the output: ./dist/index.js:

"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
/**
 * AFAIK, all envs which support Node cache modules,
 * but, just in case, you can memoize it:
 */
let unified;
async function getUnified() {
    if (typeof unified !== 'undefined')
        return unified;
    const mod = await Promise.resolve().then(() => require('unified'));
    ({ unified } = mod);
    return unified;
}
async function logUnified() {
    const unified = await getUnified();
    console.log('This is unified:', unified);
}
logUnified();

Solutions

Why is the call to require still in there? This GitHub issue ms/TS#43329 explains why TS still compiles this way, and offers two solutions:

1. In your TSConfig, set compilerOptions.module to "node12" (or nodenext).

2. If #1 is not an option (you didn't say in your question), use eval as a workaround

Let's explore both options:

Solution 1: Modify TSConfig

Let's modify the compilerOptions.module value in ./tsconfig.json:

{
  "compilerOptions": {
    ...
    "module": "node12",
    ...
  },
...
}

And run again:

$ npm run test

> [email protected] test
> npm run compile && node dist/index.js


> [email protected] compile
> tsc

tsconfig.json:8:15 - error TS4124: Compiler option 'module' of value 'node12' is unstable. Use nightly TypeScript to silence this error. Try updating with 'npm install -D typescript@next'.

8     "module": "node12",
                ~~~~~~~~


Found 1 error.

Another compiler error! Let's address it by following the suggestion in the diagnostic message: updating TS to the unstable version typescript@next:

$ npm uninstall typescript && npm install --save-dev typescript@next
--- snip ---

$ npm ls
[email protected] /so-70545129
├── @types/[email protected]
├── [email protected]
└── [email protected]

The version of typescript now installed is "^4.6.0-dev.20211231"

Let's run again:

$ npm run test

> [email protected] test
> npm run compile && node dist/index.js


> [email protected] compile
> tsc

node:internal/process/promises:246
          triggerUncaughtException(err, true /* fromPromise */);
          ^

Error [ERR_REQUIRE_ESM]: require() of ES Module /so-70545129/node_modules/unified/index.js from /so-70545129/dist/index.js not supported.
Instead change the require of /so-70545129/node_modules/unified/index.js in /so-70545129/dist/index.js to a dynamic import() which is available in all CommonJS modules.
    at /so-70545129/dist/index.js:30:65
    at async getUnified (/so-70545129/dist/index.js:30:17)
    at async logUnified (/so-70545129/dist/index.js:35:21) {
  code: 'ERR_REQUIRE_ESM'
}

Still the same error. Here's the output for examination: ./dist/index.js:

"use strict";
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
    if (k2 === undefined) k2 = k;
    Object.defineProperty(o, k2, { enumerable: true, get: function() { return m[k]; } });
}) : (function(o, m, k, k2) {
    if (k2 === undefined) k2 = k;
    o[k2] = m[k];
}));
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
    Object.defineProperty(o, "default", { enumerable: true, value: v });
}) : function(o, v) {
    o["default"] = v;
});
var __importStar = (this && this.__importStar) || function (mod) {
    if (mod && mod.__esModule) return mod;
    var result = {};
    if (mod != null) for (var k in mod) if (k !== "default" && Object.prototype.hasOwnProperty.call(mod, k)) __createBinding(result, mod, k);
    __setModuleDefault(result, mod);
    return result;
};
Object.defineProperty(exports, "__esModule", { value: true });
/**
 * AFAIK, all envs which support Node cache modules,
 * but, just in case, you can memoize it:
 */
let unified;
async function getUnified() {
    if (typeof unified !== 'undefined')
        return unified;
    const mod = await Promise.resolve().then(() => __importStar(require('unified')));
    ({ unified } = mod);
    return unified;
}
async function logUnified() {
    const unified = await getUnified();
    console.log('This is unified:', unified);
}
logUnified();

TS is still transforming the dynamic import into a call to require, even though we've followed all diagnostic message suggestions, and configured the project correctly. 😡 This seems like a bug at this point.

Let's try the workaround instead, but first, let's undo the changes we just made:

First, uninstall the unstable version of typescript and reinstall the stable one:

$ npm uninstall typescript && npm install --save-dev typescript
--- snip ---

$ npm ls
[email protected] /so-70545129
├── @types/[email protected]
├── [email protected]
└── [email protected]

The version of typescript now installed is "^4.5.4"

Then, modify the compilerOptions.module value back to "CommonJS" in ./tsconfig.json:

{
  "compilerOptions": {
    ...
    "module": "CommonJS",
    ...
  },
...
}

Solution 2: Workaround using eval

Let's modify ./src/index.ts, specifically the function getUnified (lines 16-21):

Currently, it looks like this:

async function getUnified (): Promise<Unified> {
  if (typeof unified !== 'undefined') return unified;
  const mod = await import('unified');
  ({unified} = mod);
  return unified;
}

and the problematic statement that TS refuses to stop transforming is on line 18:

const mod = await import('unified');

Let's move this into a string literal and evaluate it at runtime using eval so that TS will not transform it:

// before:
const mod = await import('unified');

// after:
const mod = await (eval(`import('unified')`) as Promise<typeof import('unified')>);

So the entire function now looks like this:

async function getUnified (): Promise<Unified> {
  if (typeof unified !== 'undefined') return unified;
  const mod = await (eval(`import('unified')`) as Promise<typeof import('unified')>);
  ({unified} = mod);
  return unified;
}

Save the file and run again:

$ npm run test

> [email protected] test
> npm run compile && node dist/index.js


> [email protected] compile
> tsc

This is unified: [Function: processor] {
  data: [Function: data],
  Parser: undefined,
  Compiler: undefined,
  freeze: [Function: freeze],
  attachers: [],
  use: [Function: use],
  parse: [Function: parse],
  stringify: [Function: stringify],
  run: [Function: run],
  runSync: [Function: runSync],
  process: [Function: process],
  processSync: [Function: processSync]
}

Finally! 🥳 The desired result is achieved. Let's compare the output one last time: ./dist/index.js:

"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
/**
 * AFAIK, all envs which support Node cache modules,
 * but, just in case, you can memoize it:
 */
let unified;
async function getUnified() {
    if (typeof unified !== 'undefined')
        return unified;
    const mod = await eval(`import('unified')`);
    ({ unified } = mod);
    return unified;
}
async function logUnified() {
    const unified = await getUnified();
    console.log('This is unified:', unified);
}
logUnified();

That's what we wanted: the dynamic import statement wasn't transformed into a require call.

Now, when you need to use the unified function, just use this syntax in your program:

const unified = await getUnified();

Elenore

thanks you so much for your effort! I read the post on dynamic import a little bit and understand that we can import ESM module in CJS. However, I also read the post on ESM vs CJS, more specifically the section CJS Can import() ESM, but It’s Not Great. In my case, I did want to export my library so that my application can use it in a sync context. I guess there is no workaround if the 3rd-party library that I am using only support ESM? – Protease

@HUIJINGHUANG Unfortunately, no. If you don’t mind sharing, what’s keeping you from writing and publishing your module as ESM? – Elenore

I see. I can publish my module as ESM. Do you have a good tutorial for me to follow? I tried to [produce both esm and commonjs](sensedeep.com/blog/posts/2021/how-to-create-single-source-npm-module.html) but in my application where I use ESM syntax now, it somehow points to dist/cjs folder instead of dist/mjs folder, which is super wired! I then tried this tutorial in order to only produce ESM module but it didn't work. – Protease

You can use as a reference the module that you want to import: unified: look at its package.json, tsconfig.json, etc. – Elenore

Thank you @jsejcksn. I finally made it work, even though in a different sense. I still find this esm vs commonjs thing very painful to deal with. – Protease

@HUIJINGHUANG 👍 If this answers your question, feel free to mark it as such so that other people will know. – Elenore

Answer 2

We were running with this issue while trying to import an ESM-only package (ChatGPT), even trying dynamically failed because import was transpiled. We found the solution in this file:

export const importDynamic = new Function('modulePath', 'return import(modulePath)');

async function fn {
  // use the dynamic import:
  const { ChatGPTAPI } = await importDynamic('chatgpt');

  // do whatever you need to do:
  api = new ChatGPTAPI();
}

Upon further thought, it seems what it is doing is simply using the ability of Javascript to evaluate a function based on a string, so the transpiler cannot touch it. Genius!

Lecce

Answer 3

I quote this part of jsejcksn's answer:

In your TSConfig, set compilerOptions.module to "node12" (or nodenext).

Currently, you can set compilerOptions.module to "node16" and it will work.

In summary, if you use dynamic imports and do the mentioned configuration it will work.

Thanks jsejcksn.

Prize

Answer 4

I don't think this workaround will apply to a lot of people, but I faced this issue with the npm package stemmer. This package happens to only contain a single index.ts file, so switching

import { stemmer } from 'stemmer';

to

import { stemmer } from '../../../../../node_modules/stemmer/index';

fixed the problem for me! I believe that using this Deep Import (which is generally discouraged...) caused webpack to skip processing of the library. Instead, the file is compiled alongside my application. But I'm no webpack expert.

Woollyheaded