How do you send HTTP/2 requests via a proxy using Node.js?

Question

I'd like to send HTTP/2 requests to a server via a proxy using Node.js's http2 library.

I'm using Charles v4.2.7 as a proxy, for testing purposes, but Charles is not able to proxy the request. Charles is showing Malformed request URL "*" errors, as the request it receives is PRI * HTTP/2.0 (the HTTP/2 Connection Preface). I can successfully sent HTTP/2 requests via my Charles proxy using cURL (e.g. curl --http2 -x localhost:8888 https://cypher.codes), so I don't think this is an issue with Charles, but instead an issue with my Node.js implementation.

Here's my Node.js HTTP/2 client implementation which tries to send a GET request to https://cypher.codes via my Charles proxy listening at http://localhost:8888:

const http2 = require('http2');

const client = http2.connect('http://localhost:8888');
client.on('error', (err) => console.error(err));

const req = client.request({
  ':scheme': 'https',
  ':method': 'GET',
  ':authority': 'cypher.codes',
  ':path': '/',
});
req.on('response', (headers, flags) => {
  for (const name in headers) {
    console.log(`${name}: ${headers[name]}`);
  }
});

req.setEncoding('utf8');
let data = '';
req.on('data', (chunk) => { data += chunk; });
req.on('end', () => {
  console.log(`\n${data}`);
  client.close();
});
req.end();

Here's the Node.js error I get when running node proxy.js (proxy.js is the file containing the above code):

events.js:200
      throw er; // Unhandled 'error' event
      ^

Error [ERR_HTTP2_ERROR]: Protocol error
    at Http2Session.onSessionInternalError (internal/http2/core.js:746:26)
Emitted 'error' event on ClientHttp2Stream instance at:
    at emitErrorNT (internal/streams/destroy.js:92:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:60:3)
    at processTicksAndRejections (internal/process/task_queues.js:81:21) {
  code: 'ERR_HTTP2_ERROR',
  errno: -505
}

I reran the above cURL request with verbose output and it looks like cURL first sends a CONNECT to the proxy using HTTP/1, before sending the GET request using HTTP/2.

$ curl -v --http2 -x localhost:8888 https://cypher.codes         
*   Trying ::1... 
* TCP_NODELAY set
* Connected to localhost (::1) port 8888 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to cypher.codes:443
> CONNECT cypher.codes:443 HTTP/1.1  
> Host: cypher.codes:443
> User-Agent: curl/7.64.1                                                                                              
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
<               
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!                                                                                             
* ALPN, offering h2                                    
* ALPN, offering http/1.1                                
* successfully set certificate verify locations:                                                                       
*   CAfile: /etc/ssl/cert.pem    
  CApath: none                                                                                                         
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* CONNECT phase completed!
* CONNECT phase completed!   
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):                                                                                                                                                                                     
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1): 
* TLSv1.2 (OUT), TLS handshake, Finished (20):                                                                                                                                                                                                
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2        
* Server certificate:                
*  subject: CN=cypher.codes
*  start date: Jun 21 04:38:35 2020 GMT
*  expire date: Sep 19 04:38:35 2020 GMT
*  subjectAltName: host "cypher.codes" matched cert's "cypher.codes"
*  issuer: CN=Charles Proxy CA (8 Oct 2018, mcypher-mbp.local); OU=https://charlesproxy.com/ssl; O=XK72 Ltd; L=Auckland; ST=Auckland; C=NZ
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7ff50d00d600)
> GET / HTTP/2
> Host: cypher.codes
> User-Agent: curl/7.64.1
> Accept: */*
> 
...

I'd like to try doing the same via Node.js (first sending a HTTP/1 CONNECT request and then sending my HTTP/2 request on the same TCP connection), but I'm not sure how to do this. The very act of creating the HTTP/2 client (i.e. http2.connect('http://localhost:8888');) sends the HTTP/2 Connection Preface. I thought about first creating a connection using HTTP/1 (e.g. using the http library) and then upgrading this to HTTP/2, but I couldn't find any examples on how to do this.

Could someone help me send a HTTP/2 request via a proxy using Node.js?

---

Update (2020-07-13): I made more progress towards first creating a connection using HTTP/1, sending a CONNECT request, and then trying to send a GET request using HTTP/2 over the same socket. I can see the CONNECT request come through in Charles, but not the additional GET request, which indicates that I'm still doing something wrong when trying to use the same socket for HTTP/2 requests. Here's my updated code:

const http = require('http');
const http2 = require('http2');

const options = {
  hostname: 'localhost',
  port: 8888,
  method: 'CONNECT',
  path: 'cypher.codes:80',
  headers: {
    Host: 'cypher.codes:80',
    'Proxy-Connection': 'Keep-Alive',
    'Connection': 'Keep-Alive',
  },
};
const connReq = http.request(options);
connReq.end();

connReq.on('connect', (_, socket) => {
  const client = http2.connect('https://cypher.codes', {
    createConnection: () => { return socket },
  });
  client.on('connect', () => console.log('http2 client connect success'));
  client.on('error', (err) => console.error(`http2 client connect error: ${err}`));

  const req = client.request({
    ':path': '/',
  });
  req.setEncoding('utf8');
  req.on('response', (headers, flags) => {
    let data = '';
    req.on('data', (chunk) => { data += chunk; });
    req.on('end', () => {
      console.log(data);
      client.close();
    });
  });
  req.end();
});

Shiloh

Answer 1

To tunnel HTTP/2 through a proxy that doesn't understand it, you need to use HTTP/1.1 for the initial connection, and then use HTTP/2 only in the tunnel. Your code uses HTTP/2 right from the start, which isn't going to work.

To actually make that tunnel, you first send an HTTP CONNECT request for the target host, and receive a 200 response, and then everything else on the connection in future is forwarded back and forth between you and the target host.

Once you have that tunnel working, you can send HTTP/2 (or anything else the target server understands) and it'll go straight to your target.

The code to do that in node looks like this:

const http = require('http');
const http2 = require('http2');

// Build a HTTP/1.1 CONNECT request for a tunnel:
const req = http.request({
  method: 'CONNECT',
  host: '127.0.0.1',
  port: 8888,
  path: 'cypher.codes'
});
req.end(); // Send it

req.on('connect', (res, socket) => {
  // When you get a successful response, use the tunnelled socket
  // to make your new request.
  const client = http2.connect('https://cypher.codes', {
    // Use your existing socket, wrapped with TLS for HTTPS:
    createConnection: () => tls.connect({
      socket: socket,
      ALPNProtocols: ['h2']
    })
  });

  // From here, use 'client' to do HTTP/2 as normal through the tunnel
});

I've been working on the internals of my own tool as well recently, to add full HTTP/2 support for proxying, and writing that up over here, which is probably super relevant for you. The tests for that in https://github.com/httptoolkit/mockttp/blob/h2/test/integration/http2.spec.ts have more & larger examples of tunnelling HTTP/2 in node like this, so those are definitely worth a look too. That's all still under development of course, so let me know if you have any questions or find any mistakes there.

Folkways

Thank you for the answer, I've encountered an issue when using a remote proxy (not localhost), here is the question: #65753859 – Walden

Answer 2

@TimPerry 's answer almost worked for me but it missed couple of things: authentication and how to avoid TLS certificate error.

So here is my updated version:

const http = require('http');
const http2 = require('http2');
const tls = require('tls');

// Build a HTTP/1.1 CONNECT request for a tunnel:
const username = '...';
const password = '...';
const req = http.request({
  method: 'CONNECT',
  host: '127.0.0.1',
  port: 8888,
  path: 'website.com', //the destination domain
  headers: { //this is how we authorize the proxy, skip it if you don't need it
    'Proxy-Authorization': 'Basic ' + Buffer.from(username + ':' + password).toString('base64')
  }
});
req.end(); // Send it

req.on('connect', (res, socket) => {
  // When you get a successful response, use the tunnelled socket to make your new request
  const client = http2.connect('https://website.com', {
    createConnection: () => tls.connect({
      host: 'website.com', //this is necessary to avoid certificate errors
      socket: socket,
      ALPNProtocols: ['h2']
    })
  });

  // From here, use 'client' to do HTTP/2 as normal through the tunnel
});

Playacting

Answer 3

I was getting an "ERR_HTTP2_ERROR" and "ERR_HTTP2_PROTOCOL_ERROR" errors with @Stalinko's answer, so i need to find an alternative...

To display my solution, we will make an request to an API that returns your IP as a JSON, then you be able to adapt to your needs.

Here is the code:

/**
 * A URL without the path.
 */
const TARGET_AUTHOTIRY = 'https://api4.my-ip.io'

/**
 * You should use the host with the port equivalent to the protocol
 * HTTP => 80
 * HTTPS => 443
 */
const TARGET_HOST = 'api4.my-ip.io:443'

/**
 * Proxy configuration
 */
const PROXY_HOST = '<your_proxy_host>'
const PROXY_PORT = '<your_proxy_port>'
const PROXY_USERNAME = '<your_proxy_username>'
const PROXY_PASSWORD = '<your_proxy_password>'

/**
 * Establishes an connection to the target server throught the HTTP/1.0
 * proxy server.
 *
 * The CONNECT method tells the PROXY server where this connection should arive.
 *
 * After the connection is established you will be able to use the TCP socket to send data
 * to the TARGET server.
 */
const request = http.request({
  method: 'CONNECT',
  host: PROXY_HOST,
  port: PROXY_PORT,
  path: TARGET_HOST,
  headers: {
    'Host': TARGET_HOST,
    'Proxy-Authorization': `Basic ${Buffer.from(`${PROXY_USERNAME}:${PROXY_PASSWORD}`).toString('base64')}`
  }
})

/**
 * Wait the "connect" event and then uses the TCP socket to proxy the HTTP/2.0 connection throught.
 */
request.on('connect', (res, socket) => {
  /**
   * Check if it has successfully connected to the server
   */
  if (res.statusCode !== 200)
    throw new Error('Connection rejected by the proxy')

  /**
   * Use the TCP socket from the HTTP/1.0 as the socket for this new connection
   * without the need to establish the TLS connection manually and handle the errors
   * manually too.
   *
   * This method accepts all TCP and TLS options.
   */
  const client = http2.connect(TARGET_AUTHOTIRY, { socket })

  client.on('connect', () => {
    console.log('Connected to the page!')
  })

  /**
   * Request to check your IP
   */
  const req = client.request({
    ':path': '/ip.json',
  })

  req.on('response', (headers) => {
    console.log('Recieved a response')
  })

  /**
   * Stores the data recieved as a response
   */
  const buffers = []
  req.on('data', (buffer) => {
    buffers.push(buffer)
  })

  req.on('end', () => {
    console.log(Buffer.concat(buffers).toString('utf-8'))

    // Closes the connection with the server
    client.close()
  })

  req.end()
})

request.end()

Instead of creating a TLS Socket, i just inject my TCP Socket in the HTTP/2.0 client.

The socket option is not explicitly listed in the method documentation, but the method accepts all net.connect() and tls.connect() options.

You can find all the documentation about http2.connect method here: HTTP 2 Node JS Documentation

Vitrain