ExpressJS Middleware bodyParser has very poor performance

Question

Few days ago we added NewRelic, APM to our Rest API, which is written in NodeJS and uses EXPRESS JS as a development framework.

Now we see a lot of users experience poor response times, because of JSON parser middleware.

Here is one of those requests reported in NewRelic:

Drilled-down report:

As you can see the most of the time is consumed by middleware JSON Parser.

We were thinking maybe issue comes from big JSON payloads, which is sometimes sent out from API. But for this response, illustrated above, API returned data with contentLength=598, which shouldn't be too huge JSON.

We also use compression middleware as visible on drilled down request execution screenshot. Which should be reducing size of IO sent back and forth to clients.

At this moment we have a doubt for a parameter limit which is passed to middleware when initialized. {limit:50kb} But when testing locally it doesn't make any difference.

We were thinking to switch with protobuf, also think about the way to parse JSON payloads asynchronously. Because JSON.parse which is used by middleware is synchronous process and stops non-blocking IO.

But before staring those changes and experiments, please if anyone had same kind of problem to suggest any possible solution.

Benchmarking:

For benchmarking on local/stage environments we use JMeter and generate loads to check when such timeouts may happen, but we are not able to catch this when testing with JMeter.

Thank You.

Ganesha

Any update on this issue ? – Voncile

Unfortunately no @Voncile – Ganesha

the slow response time happens only when new relic is used? – Mungovan

We met the same issue, we test it through response-time middleware with or without body-parser. With body-parser there is some high response time... – Carbrey

I was seeing a similar issue with the urlencodedParser in production traces. This comment seemed to have the only reasonable explanation I could find #41175819 that the cause is slow requests from the client, and the middleware blocks until the request has been fully buffered. – Lighterage

Answer 1

Express comes with an embedded bodyparser, and you can try it if you want. It should perform better since it's integrated.

const express = require('express');
const app = express();

app.use(express.urlencoded()); // support for GET

// No limited usage,
app.use(express.json());       // support for POST
// If you want add request limit,
app.use(express.json({ limit: "1mb" }));

Arredondo

For express 4.16.x, this is the same bodyParser. They just bundle it. The issue remains the same. – Khachaturian