keep-alive connection with Superagent

Question

I'm running a node.js server A which uses superagent to issue HTTP requests to another server B.

I investigated the request on server B and saw the the header connection being close and the httpVersion being 1.1:

var http = require('http');
var request = require('superagent');

http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.write('req.httpVersion seen on server:' + req.httpVersion);
  res.write('\nreq.headers.connection seen on server:' + req.headers.connection);
  res.end();
}).listen(1337, '0.0.0.0');

request
  .get('localhost:1337/helloword')
  .end(function (err, res) {
    console.log(res.text);
  });

This leads to:

req.httpVersion seen on server:1.1
req.headers.connection seen on server:close

However if I access the same server from a browser I get:

req.httpVersion seen on server:1.1
req.headers.connection seen on server:keep-alive

From https://www.rfc-editor.org/rfc/rfc2616#page-172 I learned that keep-alive is the default for HTTP 1.1 unless declared otherwise by using Connection: close.

So, my questions are:

• Why does Superagent / Node.js explicitly set the request to not use keep-alive / persistent connections?
• How can I force Superagent / Node.js to use keep-alive connections?
• How can I further influence the exact behaviour (how many connections to keep open, timeouts etc.)?

Minim

Answer 1

It doesn't seem to be documented but you can pass an http agent to superagent with the function agent. So you could create a keep-alive agent with this module: https://www.npmjs.org/package/agentkeepalive and pass it to superagent.

Something like this:

util = require('util');
util.debuglog = require('debuglog');
var http = require('http');
var request = require('superagent');
var Agent = require('agentkeepalive');

var keepaliveAgent = new Agent({
  maxSockets: 100,
  maxFreeSockets: 10,
  timeout: 60000,
  keepAliveTimeout: 30000 // free socket keepalive for 30 seconds
});

http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.write('req.httpVersion seen on server:' + req.httpVersion);
  res.write('\nreq.headers.connection seen on server:' + req.headers.connection);
  res.end();
}).listen(1337, '0.0.0.0');

request
  .get('localhost:1337/helloword')
  .agent(keepaliveAgent)
  .end(function (err, res) {
    console.log(res.text);
  });

Harwell

This does not work. It doesn't fail but superagent always creates its own agent. The param is only an options param – Marxism

Answer 2

Browers can reuse socket handle, so it send the header Connection: keep-alive to server.

If you want to keep alive connection you can send that header like this:

request
.get('localhost:1337/helloword')
.set('Connection', 'keep-alive')
.end(function (err, res) {
   console.log(res.text);
});

Haphazardly

Hm, can this really be sufficient? The receiving side 'server B' can and will handle it as a keep-alive connection. But I doubt that on the requesting side 'server A' this will be enough. In my opinion the 'server A' will still open a new connection on every request made. I'm not yet sure how to prove/verify this. – Minim

I don't know. But superagent is depends on node.js http, and node.js http support keep-alive for reusing sockets. nodejs.org/api/http.html#http_http_createserver_requestlistener – Haphazardly

@Minim What happens when you try it like this? What's the result? – Twombly

Answer 3

For superagent .timeout(5000) is also available to use.

                    await superagent.get(`${anyurl}`, { rejectUnauthorized: false })
                        .set({ "Accept": "application/json", "Content-Type": "application/json", })
                        .auth(username, password, { "type": "auto" })
                        .timeout(1000)

Ingulf