How to increase EsLint memory to avoid `JavaScript heap out of memory`?

Question

I'm trying to run EsLint on a very large javascript file and the process is running out of memory. To give you an idea of how big the file is I ran Cloc on it and here's the output:

$ cloc app.js 
       1 text file.
       1 unique file.                              
       0 files ignored.

github.com/AlDanial/cloc v 1.80  T=12.81 s (0.1 files/s, 42499.8 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
JavaScript                       1           4255          23744         516524
-------------------------------------------------------------------------------

The file weighs 23MB.

$ ls -lAh app.js 
-rw-r--r-- 1 miguelangel staff 23M Jan 28 11:58 app.js

This is all to say this is very probably not a memory leak in EsLint. I've seen some issues with EsLint having memory leaks in Github.com. I don't think that's the case here.

The file is this big because it's the result of concatenating many other Javascript modules. My aim here is to try to find any unused code. The code-base of this project has obviously grown without control and I'm trying to skim off dead weight. So I'm trying to run EsLint's no-unused-vars rule on a concatenation of the whole code-base. Here's my EsLint config file:

.eslintrc.js

module.exports = {
    "env": {
        "browser": true,
        "commonjs": false,
        "es6": true
    },
    "parserOptions": {
        "ecmaVersion": 2015
    },
    "rules": {
        "no-unused-vars": [
            "warn"
        ]
    }
};

This project is not a Node nor AMD project so I figured I had to put the entire code-base in a single file to avoid false positives.

The problem is that trying to run EsLint on this file results in a JavaScript heap out of memory error.

$ eslint app.js 

<--- Last few GCs --->

[60451:0x104002200]    43814 ms: Mark-sweep 1395.7 (1424.1) -> 1395.2 (1423.6) MB, 5719.6 / 0.0 ms  (+ 0.1 ms in 28 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 5755 ms) (average mu = 0.148, current mu = 0.037) alloca[60451:0x104002200]    49447 ms: Mark-sweep 1397.4 (1424.1) -> 1396.9 (1425.6) MB, 5569.8 / 0.0 ms  (+ 0.1 ms in 11 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 5598 ms) (average mu = 0.081, current mu = 0.011) alloca

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x3275f3d4fb7d]
Security context: 0x14c691f9d969 <JSObject>
    1: /* anonymous */ [0x14c6f1b7b981] [/usr/local/lib/node_modules/eslint/node_modules/acorn/dist/acorn.js:~2868] [pc=0x3275f40f5843](this=0x14c6b794c669 <Parser map = 0x14c603088f11>)
    2: /* anonymous */ [0x14c6f1b7b111] [/usr/local/lib/node_modules/eslint/node_modules/acorn/dist/acorn.js:2190] [bytecode=0x14c691fecb01 offset=968](this=0x14c6b794c669 <...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x10003ace0 node::Abort() [/usr/local/bin/node]
 2: 0x10003aeb1 node::OnFatalError(char const*, char const*) [/usr/local/bin/node]
 3: 0x10018c8cf v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 4: 0x10018c870 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 5: 0x10047b188 v8::internal::Heap::UpdateSurvivalStatistics(int) [/usr/local/bin/node]
 6: 0x10047cc01 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/usr/local/bin/node]
 7: 0x10047a4c4 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/bin/node]
 8: 0x100479236 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
 9: 0x100481826 v8::internal::Heap::AllocateRawWithLightRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/bin/node]
10: 0x100481b5c v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/bin/node]
11: 0x100461562 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/usr/local/bin/node]
12: 0x100653464 v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
13: 0x3275f3d4fb7d 
Abort trap: 6

How can I increase EsLint's access to memory?

Mcinerney

The Closure Compiler by Google may be better adapt to your situation. You can add declarations, a bit like a #include in C/C++, see the @externs. That way you can still run against 100% proper definitions. I used it as a linter only and it's really good at finding problems that even eslint doesn't find. And you can break down the code in modules or projects and still get fully typed checks. – Hyaline

@AlexisWilke can you please show or point to an example of how the Closure Compiler can be used to flag unused code without removing it? I don't just want this removed from the final distribution. I want to clean the code-base. – Mcinerney

Closure with ADVANCED_OPTIMIZATIONS will compile unused code out. That's not automatically practical for that. It may change your function names, and sure makes it difficult to find the functions... Assuming you have the time to write tests, I would instead recommend using Istanbul (nyc) to have a coverage test. Whatever is never covered is probably dead code. – Hyaline

An interesting read about such: https://mcmap.net/q/265415/-find-dead-javascript-code – Hyaline

I think you can use the --jscomp_error=uselessCode command line option so you get errors for any function that isn't explicitly called. – Hyaline

Right, so, about that other thread you mentioned, this is exactly what I'm doing, except that guy is proposing JSLint instead of ESLint. I think the Closure Compiler would be nice if there were a way to run it just to flag the unused code without minifying or removing anything. I looked for the option but if there is one I couldn't find it. – Mcinerney

The uselessCode warning seems to be the one you're looking for. On my end I don't use the output of the Closure compiler. Instead I use it more or less like a static analyzer which tells me things such as "you're calling this function with a string instead of an array" which I think is quite useful. – Hyaline

Answer 1

I've browsed the EsLint docs hoping to find an option to configure it to run with more memory. Sadly I was not able to find anything of the sort.

However, inspired by @vsemozhetbyt's answer -hence the up-vote- I started looking at Node's configuration options and I've found a way to get around this using Node's NODE_OPTIONS environment variable.

A space-separated list of command line options. options... are interpreted as if they had been specified on the command line before the actual command line

So this is what I did:

$ export NODE_OPTIONS="--max-old-space-size=4096"
$ echo $NODE_OPTIONS
--max-old-space-size=4096
$ eslint app.js

This way the Node process that runs EsLint just picks this up without the need for me to type in the --max-old-space-size flag nor the path to the Node binary every time I invoke EsLint.

Mcinerney

Answer 2

By default, Node.js sets memory limit near 1.5 GB. You can increase it by --max_old_space_size key (say, --max_old_space_size=4096 to use 4 GB).

To use the key, you need to know the path to the ESLint main file and call it as a script this way:

node --max_old_space_size=4096 ./node_modules/eslint/bin/eslint.js app.js

Aguilar

This is a good workaround but I was hoping for something less verbose and which could be done just once. – Mcinerney

add it in the scripts of your package.json, then just run npn run lint when you want to lint – Godevil

Try this instead in-case eslint moves their file locations node --max_old_space_size=4096 ./node_modules/.bin/eslint app.js – Smaze

Answer 3

I was having this problem in a Laravel/Vue application. Those details aren't significant, but the fact that Laravel has some folders such as vendor that includes all the composer packages meant that ES Lint was traversing a huge number of files, enough to blow out the heap.

The solution was to first add a .eslintignore file into the root of the project, such as:

/app
/bootstrap
/config
/database
/node_modules
/public
/routes
/storage
/tests
/vendor

Next, to execute ES Lint from the CLI (and GitLab-CI) as:

node_modules/eslint/bin/eslint.js --config ./.eslintrc.json .

It was also important to make sure ES Lint was using the correct config file to check against the intended ruleset.

You can visually test against some of this by adding --debug onto your CLI command, and you will see it traversing illegal folders and files if that is occurring. I think that is a decent debug step--to test it in verbose mode. You might see ES Lint traversing unnecessary folders or testing against incorrect rules.

In my opinion, this problem can easily stem from ES Lint trying to load a huge number of errors into memory while it is working. Rather than increase the heap, you can dial in what exactly ES Lint is linting.

Outcaste

My project had an eslintignore file, but a recent change had created a new folder full of build files. This new folder was not added to the ignore list, so this was causing an issue. Thanks for the reminder! – Iolaiolande

I think this solution should actually be explored first before trying to increase the memory available to Node.js. I'm saying this because I simply added a .eslintignore to my project, ignored node_modules and some other folders and ESLint stopped running out of memory. – Ivers

I agree because, if you imagine the scenario as a glass of water that can be filled, adding more height to the glass does solve it, but it doesn't solve the original problem of why water is going that high to begin with. It is important to make sure your project understand what to lint and what to avoid linting. – Outcaste

Once I ignored all the build folders, it fixed it for me – Beefcake

Answer 4

Put a .env file in your folder, and with content:

NODE_OPTIONS=--max_old_space_size=4096

Tailwind

Answer 5

If all answers about increase Node --max_old_space_size will not help it's worth taking a look at it from a different angle. Maybe there are so many errors that Node heap is really out of memory.

I had this situation:

I joined to big project with a lot of .js/.vue files. In .eslintrc was "linebreak-style": [2, "unix"] that means end of line character should be lf.I use Windows where eol character is crlf. Git by default converts everything to crlf. So when I cloned this project from git I saw a lot of warnings from Eslint about wrong eol character in Visual Studio Code but eslint from console always ended with heap out of memory.

For above situation none of advices about node out of memory did not help cause problem was elsewhere.

Monolayer

Thank you! Had 1000+ errors, which finally showed when I upgraded typescript to v3.9.3. It has better build time so eslint benefits from it too. When resolving the errors, had to downgrade typescript again to make it work with Angular compiler. But problem solved! – Colenecoleopteran

Answer 6

I've got the same error after migration from TSLint to ESLint. In my case typescript configurataion was a bottleneck. This line in tslint.json file:

"extends": "tslint-consistent-codestyle"

With this extend each file was analyzed dozen seconds (out of memory error after 2-3 minutes). Without it all project (hundred files) was analized in few seconds.

Furnace