Node.js error: too many parameters Error while uploading bulk data

Question

I have a task to upload user data in bulk through csv file. I am using nodejs and express framework. When i submit csv file having 60 to 70 rows it works fine, but when it exceeds 70 rows it starts giving server error too many parameters. After some research i concluded that it may be the problem of body parser size, so i tried This blog , but it didnt worked error is still same.

here is my code for body parser:

var cookieParser = require('cookie-parser');
var bodyParser = require('body-parser');
app.use(cookieParser());
app.use(bodyParser({limit: '50mb'}));
app.use(bodyParser.json({limit: '50mb'}));
app.use(bodyParser.urlencoded({ extended: false }));

Error message:

2016-04-19T10:29:45.299Z - error: [req#d3a1fa1a-278e-496e-9cb1-b3a944e3d1c8/app] [App] Error: too many parameters Error: too many parameters
    at queryparse (d:\Git\gap-vm 13416\node_modules\body-parser\lib\types\urlencoded.js:119:17)
    at parse (d:\Git\gap-vm 13416\node_modules\body-parser\lib\types\urlencoded.js:64:9)
    at d:\Git\gap-vm 13416\node_modules\body-parser\lib\read.js:91:18
    at IncomingMessage.onEnd (d:\Git\gap-vm 13416\node_modules\raw-body\index.js:136:7)
    at IncomingMessage.g (events.js:273:16)
    at emitNone (events.js:80:13)
    at IncomingMessage.emit (events.js:179:7)
    at endReadableNT (_stream_readable.js:906:12)
    at nextTickCallbackWith2Args (node.js:474:9)
    at process._tickCallback (node.js:388:17)

So , can anyone tell me where i am going wrong. Any suggestion would be helpful. Thanx in advance.

Curtice

have you find any solution? – Primordial

Answer 1

As others mention, you'll need to set the parameterLimit to deal with the "too many parameters" error. You may also need to set the limit to a larger size to avoid a load size error. In the case of CSV, the urlencoded limits will be applied, but others may also want to set the JSON limits too. The following setting will work unless there are other places in the code that are overriding these settings:

var bodyParser = require('body-parser');
app.use(bodyParser.json({limit: '50mb'}));
app.use(bodyParser.urlencoded({limit: '50mb', extended: true, parameterLimit: 1000000}));

Anastasia

Set parameterLimit as option of bodyParser.json() is wrong. Look github.com/expressjs/body-parser#bodyparserjsonoptions – Jasminjasmina

Agreed. I removed parameterLimit from bodyParser.json() – Anastasia

Answer 2

In your code it you are not using the parameterLimitat all, as pointed out as in the blog posted you linked.

var bodyParser = require('body-parser');
app.use(bodyParser.urlencoded({ 
    extended: false,
    parameterLimit: 1000000 // experiment with this parameter and tweak
}));

Fifth

@ Francesco actually i tried this code before , but it was not working so i reverted it to original.Above code not working . – Curtice

Did you try setting both limitand parameterLimit for your bodyParser.urlencoded middleware? According to body-parser docs, there are only 5 settings available: inflate, limit, parameterLimit, type, verify – Fifth

Answer 3

I'm not sure where you guys are testing your API but for me it was because I set the Content-Type header to application/x-www-form-urlencoded in Postman. Once I removed the header and used form-data under the body section, it solved the issue. Make sure to always use form-data when uploading files. Hope it helps...

Beckwith