Node.js - get raw request body using Express [duplicate]

Question

When I use Express, and my code is:

app.use(express.bodyParser());

How would I get the raw request body?

Journeyman

Answer 1

Edit 2: Release 1.15.2 of the body parser module introduces raw mode, which returns the body as a Buffer. By default, it also automatically handles deflate and gzip decompression. Example usage:

var bodyParser = require('body-parser');
app.use(bodyParser.raw(options));

app.get(path, function(req, res) {
  // req.body is a Buffer object
});

By default, the options object has the following default options:

var options = {
  inflate: true,
  limit: '100kb',
  type: 'application/octet-stream'
};

If you want your raw parser to parse other MIME types other than application/octet-stream, you will need to change it here. It will also support wildcard matching such as */* or */application.

---

Note: The following answer is for versions before Express 4, where middleware was still bundled with the framework. The modern equivalent is the body-parser module, which must be installed separately.

The rawBody property in Express was once available, but removed since version 1.5.1. To get the raw request body, you have to put in some middleware before using the bodyParser. You can also read a GitHub discussion about it here.

app.use(function(req, res, next) {
  req.rawBody = '';
  req.setEncoding('utf8');

  req.on('data', function(chunk) { 
    req.rawBody += chunk;
  });

  req.on('end', function() {
    next();
  });
});
app.use(express.bodyParser());

That middleware will read from the actual data stream, and store it in the rawBody property of the request. You can then access the raw body like this:

app.post('/', function(req, res) {
  // do something with req.rawBody
  // use req.body for the parsed body
});

Edit: It seems that this method and bodyParser refuse to coexist, because one will consume the request stream before the other, leading to whichever one is second to never fire end, thus never calling next(), and hanging your application.

The simplest solution would most likely be to modify the source of bodyParser, which you would find on line 57 of Connect's JSON parser. This is what the modified version would look like.

var buf = '';
req.setEncoding('utf8');
req.on('data', function(chunk){ buf += chunk });
req.on('end', function() {
  req.rawBody = buf;
  var first = buf.trim()[0];
  ...
});

You would find the file at this location:

/node_modules/express/node_modules/connect/lib/middleware/json.js.

Seena

It seems we can not do this. If I added these code, the events in \node_modules\express\node_modules\connect\lib\middleware\urlencoded.js will not fired. req.on("data"),req.on("end") not fired. – Journeyman

After added your code, I process my post use this code app.post("/ajax", function(req,res) { res.send('hello world,post'); } ); when my request's content-type is application/x-www-form-urlencoded, server will not response "hello world,post" – Journeyman

the location of the file is incorrect in my case.. I don't have a connect module in the node_modules of express (>4.0.0) haven't found the new location yet – Grit

Connect is not a dependency in Express 4, and therefore does not contain the Body Parser module. If you still needed it, you would find it here. – Seena

@Seena Can you please update your answer and include a reference to the newest body-parser middleware? The module now includes the raw-body-parser middleware. This can be helpful for googlers looking for a method to get the raw body. – Stedt

express.raw wasn't working for me with Google Cloud Tasks, because Tasks wasn't sending a content-type header with http tasks. I was able to manually specify the header and it worked from there. Got back a Buffer which I converted to JSON with some custom middleware. – Celiaceliac

Answer 2

This solution worked for me:

var rawBodySaver = function (req, res, buf, encoding) {
  if (buf && buf.length) {
    req.rawBody = buf.toString(encoding || 'utf8');
  }
}

app.use(bodyParser.json({ verify: rawBodySaver }));
app.use(bodyParser.urlencoded({ verify: rawBodySaver, extended: true }));
app.use(bodyParser.raw({ verify: rawBodySaver, type: '*/*' }));

When I use solution with req.on('data', function(chunk) { }); it not working on chunked request body.

Floating

This took care of every major scenario of incoming data in various parts of the request. – Serenity

I was attempting to verify the hmac for a Shopify app webhook, and this worked for me. I roughly followed this example: gist.github.com/andjosh/5c4f0244914adfd312e4. – Advisee

This make sure you can still read req.body as Buffer when content-type is specified as 'application/octet-stream' which is very straighfoward when working with JSON post and other binary files post. – Preface

This solution worked for me but it doesn't work for multipart request, for that we need to remove below line. app.use(bodyParser.raw({ verify: rawBodySaver, type: '/' })); – Jacket

Answer 3

I got a solution that plays nice with bodyParser, using the verify callback in bodyParser. In this code, I am using it to get a sha1 of the content and also getting the raw body.

app.use(bodyParser.json({
    verify: function(req, res, buf, encoding) {

        // sha1 content
        var hash = crypto.createHash('sha1');
        hash.update(buf);
        req.hasha = hash.digest('hex');
        console.log("hash", req.hasha);

        // get rawBody        
        req.rawBody = buf.toString();
        console.log("rawBody", req.rawBody);

    }
}));

I am new in Node.js and express.js (started yesterday, literally!) so I'd like to hear comments on this solution.

Lifesaver

I really like this solution. I just included req.rawBody = buf.toString(); and took the rest out of the verify function, because that was all I needed, and it worked beautifully. No need to change the bodyParser source code! – Stun

+1 but my problem now is that I need an async function to verify wether this request was previously sent or not :/ – Sanatory

very nice. may I suggest req.rawBody = buf.toString(encoding); – Ladew

This will capture only application/json requests – Floating

Answer 4

BE CAREFUL with those other answers as they will not play properly with bodyParser if you're looking to also support json, urlencoded, etc. To get it to work with bodyParser you should condition your handler to only register on the Content-Type header(s) you care about, just like bodyParser itself does.

To get the raw body content of a request with Content-Type: "text/plain" into req.rawBody you can do:

app.use(function(req, res, next) {
  var contentType = req.headers['content-type'] || ''
    , mime = contentType.split(';')[0];

  if (mime != 'text/plain') {
    return next();
  }

  var data = '';
  req.setEncoding('utf8');
  req.on('data', function(chunk) {
    data += chunk;
  });
  req.on('end', function() {
    req.rawBody = data;
    next();
  });
});

Trying

+1. I tried one of the solutions above and then all of my GETs and json posts were failing. The solutions above are technically correct for the question, but if you're handling more diverse requests with data in more than one form, you'll need this. – Adenoid

What is data here? is it the variable we are sending from the UI? – Shaker

app.use(bodyParser.urlencoded({limit: '80mb', extended: true})); app.use(bodyParser.json({limit: '80mb'})); app.use(bodyParser.raw({type: 'application/octet-stream'})) This would also do. – Talavera

Answer 5

In 2022

The best way to get raw body in every API is to convert buffer into a string.

app.use(
  express.json({
    limit: '5mb',
    verify: (req, res, buf) => {
      req.rawBody = buf.toString();
    },
  })
);

Abrasion

elegant and works for me. No need for middleware and doesn't interfere with other req objects. – Orbadiah

Agreed, this works great for fixing stripe webhook integration! – Hubsher

This ended many days of searching for a solution to matching hashes on an authorize.net webhook integration Thank you. – Entebbe

As above, I tried many different things and searched for an answer to this issue for days. This works, thank you. – Wahhabi

Answer 6

This is a variation on hexacyanide's answer above. This middleware also handles the 'data' event but does not wait for the data to be consumed before calling 'next'. This way both this middleware and bodyParser may coexist, consuming the stream in parallel.

app.use(function(req, res, next) {
  req.rawBody = '';
  req.setEncoding('utf8');

  req.on('data', function(chunk) { 
    req.rawBody += chunk;
  });

  next();
});
app.use(express.bodyParser());

Entry

This doesn't seem to work on long bodies, which get cut off early. – Knowing

Worked perfectly, saved me. Thank you. – Triptych

I am confirming that this works for large files also. I tried sending a text file of 1.5MB and the whole data was received properly. Thank you – Cowslip

@AdamLockhart - what size were your requests that got cut? – Impudent

@UpTheCreek, been a while. Not sure. My latest stuff doesn't use this snippet, but if others report no problems it may have been a bug that has been fixed. – Knowing

The whole request will be there on req.on('end', ... ). You need to wait until that if you want the full request – Prosthesis

Answer 7

// Change the way body-parser is used
const bodyParser = require('body-parser');

var rawBodySaver = function (req, res, buf, encoding) {
  if (buf && buf.length) {
    req.rawBody = buf.toString(encoding || 'utf8');
  }
}
app.use(bodyParser.json({ verify: rawBodySaver, extended: true }));




// Now we can access raw-body any where in out application as follows
request.rawBody;

Deragon

Answer 8

I have resolved the adding to the header property.

{"Content-Type": "application/json"}

Fatuitous

Answer 9

Use

app.use(express.json());

or

app.use(express.text());

or

app.use(express.urlencoded());

Depending upon your raw format

Irritation

These are literally the opposite of raw request body. – Hagan

Answer 10

Use body-parser Parse the body with what it will be:

app.use(bodyParser.text());

app.use(bodyParser.urlencoded());

app.use(bodyParser.raw());

app.use(bodyParser.json());

ie. If you are supposed to get raw text file, run .text().

Thats what body-parser currently supports

Fadiman