configuration max old space size in Nodejs

Question

Im a newbie in nodejs and mongodb. I have a problem when i read about 100000 records from my mongodb in nodejs application. When I try to get 100000 records, I got this error:

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory

I search google and everyone said that configure max old space size parameter because v8 engine have about 1,9Gb heap memmory.

My point is, I run my app by eclispe, and i dont know how to configure max-old-space-size parameter.

Can anybody give a hint?

Thank you so much!

ps: my english is bad so if you cannot understand my question, it's fine.

Severity

Why are you trying to store all of those records in memory? – Glogau

I want to display all of these records on my website. – Severity

So why not stream the records on to the browser or implement some kind of paging (browsers don't always handle a ton of dom elements that well, assuming that's how you're displaying the data)? – Glogau

I just run code query = db.Access.find().limit(limit);query.exec(function (err, data) { ... }, and my app throw FATAL ERROR. – Severity

@mscdex: thank you for your help. – Severity

Answer 1

Use node --max_old_space_size=5000 yourapp.js to allow 5000 mb. Look in your eclipse launch settings for command-line parameters, where you can add this.

However, as the comments say, you should reconsider loading this into memory, and stream it to the client instead.

Picoline

thank you for your comment. as you said "loading this into memory, and stream it to the client". can you show me docs or whatever can help to do that, because i have no idea to start. – Severity

Check out this question: #7373126 – Picoline

is it "--max_old_space_size=5000" or "--max-old-space-size=5000" ?! this is confusing – Protist

@Picoline according to the node documentation: v8 options allow words to be separated by both dashes (-) or underscores (_). For example, --stack-trace-limit is equivalent to --stack_trace_limit. (from nodejs.org/api/cli.html#cli_v8_options) – Shipwreck

do we have to run this command everytime i run the node in Terminal or once run, sets forever? – Cobb

@Cobb Yes, you have to include this option each time you run the node command – Nyala

Answer 2

you could add the following to your package.json

"scripts": {
    "start": "node --max-old-space-size=102400 ./app.js",
     ......
}

this is unlike the other way has to write only once and configured each run time. Note: you no need to run explicitly this command the npm start will take care of configuring it each time

Coppins

The default value of --max-old-space-size is 1024, i.e. 1GB. You're giving it over 100GB of memory! – Spirillum