Node.js: how to limit the HTTP request size and upload file size?

Question

I'm using Node.js and express.

I would like to limit the HTTP request size. Let's say, if someone sends me a HTTP request more than 2 MB then I stop the request right away. I looked at the code and I think if I change the core, I can do it. However, is there a way to set a max_request_size or soemthing like that?

It is kind of related to my second question. I'm using express to get an uploaded file from req.files. Is there a way to stop writing the file to the /tmp folder (which is the default upload behavior) as soon as the file size exceeds a certain file size?

Safeconduct

Answer 1

Just an update (07-2014), as I'm not able to add comments:

As correctly noted above, newer Express versions have deprecated the use of the limit middleware and now provide it as a built-in option for the BodyParser middleware:

   var express    = require('express')
   var bodyParser = require('body-parser')

   var app = express()
   app.use(bodyParser.json({ limit: '5mb' }))

Sipes

how can I increase limit in case of request module – Sculpture

What if the size of the request is not in the body? :( What if someone starts sending 50Mb GET requests? :( – Valenti

Anything not in the body will be in the headers and NodeJS sets a default limit on the maximum header size. For example, in NodeJS version 11.6.0 the limit is 8kb. nodejs.org/api/http.html#http_http_maxheadersize – Gradey

how i am gonna use it with react js ? – Formfitting

This will only limit the request body for requests having having a Content-Type of application/json. I want to limit any request, including the ones I'm not buffering. – Horizon

@Horizon you'll need to use bodyParser.raw in that case. – Sipes

Answer 2

Source code of node github :

/* Maximium header size allowed. If the macro is not defined
 * before including this header then the default is used. To
 * change the maximum header size, define the macro in the build
 * environment (e.g. -DHTTP_MAX_HEADER_SIZE=<value>). To remove
 * the effective limit on the size of the header, define the macro
 * to a very large number (e.g. -DHTTP_MAX_HEADER_SIZE=0x7fffffff)
 */
#ifndef HTTP_MAX_HEADER_SIZE
# define HTTP_MAX_HEADER_SIZE (80*1024)
#endif 

So, you need to rebuild node from source to surpass the limit of 80*1024

You can use this with Express 4 to limit request body size/Upload file size, instead of express.json() and express.urlencoded(), you must require the body-parser module and use its json() and urlencoded() methods, if the extended option is not explicitly defined for bodyParser.urlencoded(), it will throw a warning (body-parser deprecated undefined extended: provide extended option).

var bodyParser = require('body-parser');
app.use(bodyParser.json({limit: '50mb'}));
app.use(bodyParser.urlencoded({limit: '50mb', extended: true}));

Histrionic

Question #1 (and my emphasis in the bounty) is about the request size. But your answer is about the request body size. It is explicitly mentioned in the bodyParser docs that the limit parameter "Controls the maximum request body size." – Valenti

Source code of node github : /* Maximium header size allowed. If the macro is not defined * before including this header then the default is used. To * change the maximum header size, define the macro in the build * environment (e.g. -DHTTP_MAX_HEADER_SIZE=<value>). To remove * the effective limit on the size of the header, define the macro * to a very large number (e.g. -DHTTP_MAX_HEADER_SIZE=0x7fffffff) */ #ifndef HTTP_MAX_HEADER_SIZE # define HTTP_MAX_HEADER_SIZE (80*1024) #endif ... So, you need to rebuild node from source to surpass the limit of 80*1024 – Histrionic

Answer 3

Express uses Connect, which has a limit middleware available. You can use this in your Express app by doing something like this:

app.use(express.limit('2mb'));

That, for example, would limit all HTTP requests to 2 MB. Since uploaded files are part of the HTTP request, any file upload larger than 2 MB would be aborted as well.

---

NOTE: This middleware is deprecated and will soon be removed. Discussion on why this is the case is available at: https://github.com/senchalabs/connect/pull/925#issuecomment-26990726

Metronome

if I limit to 2MB, and I send over a http request of 1GB. Will it stop right away if it exceed 2MB? or returns error but still getting the rest of the 0.998GB data? – Safeconduct

The source is available at github.com/senchalabs/connect/blob/master/lib/middleware/…. If I'm reading it correctly, it reads chunked data and terminates the request as soon as it exceeds the size limit. – Metronome

express.limit is deprecated on nodejs 6 as middleware – Barbarous

Answer 4

For an updated solution that isn't deprecated, you can add the limit like so, in your app.js file:

app.use(express.json({limit: '2mb'}));
app.use(express.urlencoded({limit: '2mb', extended: false}));

You can also do it like this:

app.use(express.json({limit: 2000000}));
app.use(express.urlencoded({limit: 2000000, extended: false}));

Economics

Answer 5

var methodOverride = require('method-override');
app.use(bodyParser.urlencoded({
        extended: true, limit: '50mb'
    }));
app.use(bodyParser.json({limit: '50mb'}));
app.use(methodOverride());

Simar

Answer 6

Use raw-body. Most middleware (like limit) is no longer bundled with Express and must be installed separately. Here's an example.

var getRawBody = require('raw-body')
app.use(function (req, res, next) {
      getRawBody(
        stream = req,
        options = {
          length: req.headers['content-length'],
          limit: '100mb',
        },
        callback = function (err, reslt) {
          if (err) {
            return next(err)
          }
          next();
          console.log(req)
        })
    })

• Remember to add the router after the app.use

Express no longer allows you to explicitly set the limit using the traditional bundled middleware as show below.

app.use(express.limit('4M'));

Alf

Answer 7

Answer for Question 1:

To limit the HTTP request size and upload file size we need to set the body-parser limit.

app.use(bodyParser.urlencoded({limit: '50mb',extended: true}));
app.use(bodyParser.json({limit: '50mb'}));

bodyParser.urlencoded

The files from the front end comes as urlencoded bodies.

Returns middleware that only parses urlencoded bodies. This parser accepts only UTF-8 encoding of the body and supports automatic inflation of gzip and deflate encodings.

A new body object containing the parsed data is populated on the request object after the middleware (i.e. req.body). This object will contain key-value pairs, where the value can be a string or array (when extended is false), or any type (when extended is true).

bodyParser.json

Returns middleware that only parses json. This parser accepts any Unicode encoding of the body and supports automatic inflation of gzip and deflate encodings.

A new body object containing the parsed data is populated on the request object after the middleware (i.e. req.body).

Note: By default the input limit for body parser is 100kb

Answer for Question 2:

To change the default upload directory we can use the following.

app.set('uploadDir', './files');  // Sets the upload Directory to files folder in your project.

Other Implementation

While including the bodyParser into the app we can mention the upload directory.

app.use(express.bodyParser({uploadDir:'./files', keepExtensions: true}));

Reference:

• https://github.com/expressjs/body-parser#limit
• https://github.com/expressjs/body-parser#bodyparserjsonoptions
• https://github.com/expressjs/body-parser#bodyparserurlencodedoptions

Issues: https://github.com/expressjs/express/issues/1684

Hope this helps!

Mixon

Question #1 (and my emphasis in the bounty) is about the request size. But your answer is about the request body size. It is explicitly mentioned in the reference links you provide: "Controls the maximum request body size." – Valenti