HTTPs proxy server only works in SwitchOmega
Asked Answered
S

2

14

I did quite a lot search and pratical trials before asking this question.

Long story:

I found a (non-English)tutorial about how to write a http proxy with Node.js.

So far what I've known and tried:

  • A HTTP proxy can handle both HTTP request and HTTPS request, but in different ways. It handles HTTP request by reading the client's request and make a new request to the target and return the response to the client. As for HTTPS request, it's dealt with a HTTP Tunnel.

enter image description here

  • The SSL proxy field in Firefox proxy settings and the Secure field in IE proxy settings (Windows) are all about setting the HTTP Tunnel. If a SSL proxy or Secure proxy is set, when a brower wants to connect to a https site, it sends a CONNECT request instead of an ordinary request.

Problems:

The CONNECT request is plain text, so firewalls can see what host I want to connect to and cut the connection. So I was thinking whether I can use https to talk to the proxy server from the very beginning. I read all related posts, but couldn't find an answer directly talking about this. And some answers also say "There's no such thing as a https proxy server".

But the tutorial says this can be done (HTTPS between client and proxy server and nothing else changes). So I give it try. I changed the server into https with my website's certificate. But eventually it only works with Proxy SwitchOmega in Chrome. It doesn't work in traditional settings like in Firefox proxy or IE proxy settings.

Proxy SwitchOmega setting:

Scheme|Protocol|Server|Port
....  | https  | .... |...

I have to select https protocol here, if I starts the https server. similarly, I have to select http protocol, if I starts the http server. Also I don't know what this protocol field stands for.


To sum it up:

proxy server | Firefox proxy setting |work? | SwitchOmega setting |work?|
 http        | http + ssl setting    | yes  | protocol http       |yes  |
 https       | http + ssl setting    | no   | protocol https      |yes  |
 https       |      -                |  -   | protocal http       |no   |

So my questions are:

  1. Can I connect to the https proxy server through the ordinary way(without an extension)? If possible, how?
  2. Why can I connect to the https proxy server through SwitchOmega?
  3. I think I build a https proxy server. But why others are saying that "There's no such thing as a https proxy server?

Source code

https server

var http = require('http');
var https = require('https');
var fs = require('fs');
var net = require('net');
var url = require('url');

console.log("qqqqq2");

function request(cReq, cRes) {
    console.log("request=====start");
    console.log(cReq.headers);
    console.log(cReq.url);
    console.log(cReq.method);
    console.log("request=====end");
    var u = url.parse(cReq.url);

    var options = {
        hostname : u.hostname, 
        port     : u.port || 80,
        path     : u.path,       
        method     : cReq.method,
        headers     : cReq.headers
    };

    var pReq = http.request(options, function(pRes) {
        cRes.writeHead(pRes.statusCode, pRes.headers);
        pRes.pipe(cRes);
    }).on('error', function(e) {
        cRes.end();
    });

    cReq.pipe(pReq);
    // console.log(cReq.headers);
    // console.log(cReq.method);
    // console.log(cReq.url);
    // console.log("^_^^_^^_^^_^^_^^_^");
    // cRes.writeHead('200');
    // cRes.end('hello world2222\n');
}

function connect(cReq, cSock) {
    console.log("connect=====start");
    console.log(cReq.headers);
    console.log(cReq.url);
    console.log(cReq.method);
    console.log("connect=====end");
    var u = url.parse('http://' + cReq.url);

    var pSock = net.connect(u.port, u.hostname, function() {
        cSock.write('HTTP/1.1 200 Connection Established\r\n\r\n');
        pSock.pipe(cSock);
    }).on('error', function(e) {
        cSock.end();
    });

    cSock.pipe(pSock);
}

var options = {
    key: fs.readFileSync('./privkey1.pem'),
    cert: fs.readFileSync('./fullchain1.pem')
};

https.createServer(options)
    .on('request', request)
    .on('connect', connect)
    .listen(9999, '0.0.0.0');

http server

var http = require('http');
var net = require('net');
var url = require('url');

console.log('qqqqq2');

function request(cReq, cRes) {
    console.log("request=====start");
    console.log(cReq.headers);
    console.log(cReq.url);
    console.log(cReq.method);
    console.log("request=====end");

    var u = url.parse(cReq.url);

    var options = {
        hostname : u.hostname, 
        port     : u.port || 80,
        path     : u.path,       
        method     : cReq.method,
        headers     : cReq.headers
    };

    var pReq = http.request(options, function(pRes) {
        cRes.writeHead(pRes.statusCode, pRes.headers);
        pRes.pipe(cRes);
    }).on('error', function(e) {
        cRes.end();
    });

    cReq.pipe(pReq);
}

function connect(cReq, cSock) {
    console.log("connect=====start");
    console.log(cReq.headers);
    console.log(cReq.url);
    console.log(cReq.method);
    console.log("connect=====end");
    var u = url.parse('http://' + cReq.url);

    var pSock = net.connect(u.port, u.hostname, function() {
        cSock.write('HTTP/1.1 200 Connection Established\r\n\r\n');
        pSock.pipe(cSock);
    }).on('error', function(e) {
        cSock.end();
    });

    cSock.pipe(pSock);
}

http.createServer()
    .on('request', request)
    .on('connect', connect)
    .listen(9999, '0.0.0.0');

Test Server

You can easily build a http proxy server and test it. But it may be cumbersome to build a https proxy server, because you need to deploy certificates. So a https proxy test server is provided, based on the code above.

Test server is deleted since I've found the answer.

Spradling answered 11/7, 2019 at 5:11 Comment(4)
I just notice that it may be the pac file that do the tricks. The SwitchOmega interface seems to merely act as a front end GUI to the pac file setting. I am going to read something about the pac file.Spradling
And Manual proxy configuration and Automatic proxy configuration URL in browsers may be 2 different kinds of settings. I used to think that one is the subset of the other.Spradling
for firefox did you try network.proxy.proxy_over_tls? Also did you see this page? "Squid can accept regular proxy traffic using https_port in the same way Squid does it using an http_port directive. Unfortunately, popular modern browsers do not permit configuration of TLS/SSL encrypted proxy connections. There are open bug reports against most of those browsers now, waiting for support to appear. If you have any interest, please assist browser teams with getting that to happen. "Agueda
@Agueda Ah Thank you Peter. I aslo got to know that link from a security stackexchange post and I've read that. I think I've found the answer. As that page describes, https proxy is supported but only via pac file not GUI (neither Chrome nor Firefox). I will make an answer to my question later.Spradling
S
10

I found the answer in Security StackExchange. Is it possible to connect to a proxy with an ssl (or otherwise encrypted) connection?

From https://wiki.squid-cache.org/Features/HTTPS#Encrypted_browser-Squid_connection :

Encrypted browser-Squid connection

While HTTPS design efforts were focused on end-to-end communication, it would also be nice to be able to encrypt the browser-to-proxy connection (without creating a CONNECT tunnel that blocks Squid from accessing and caching content). This would allow, for example, a secure use of remote proxies located across a possibly hostile network.

Squid can accept regular proxy traffic using https_port in the same way Squid does it using an http_port directive. Unfortunately, popular modern browsers do not permit configuration of TLS/SSL encrypted proxy connections. There are open bug reports against most of those browsers now, waiting for support to appear. If you have any interest, please assist browser teams with getting that to happen.

...

Chrome

The Chrome browser is able to connect to proxies over SSL connections if configured to use one in a PAC file or command line switch. GUI configuration appears not to be possible (yet).

Firefox

The Firefox 33.0 browser is able to connect to proxies over TLS connections if configured to use one in a PAC file. GUI configuration appears not to be possible (yet), though there is a config hack for embedding PAC logic.

More information related to Chrome can be found in http://dev.chromium.org/developers/design-documents/secure-web-proxy.


To answer the questions:

  1. Can I connect to the https proxy server through the ordinary way(without an extension)? If possible, how?

The traditional way(e.g. Manual proxy configuration field in Firefox) to set a http proxy server is for HTTP proxy server only. One can only set a https proxy via pac files (e.g. Automatic proxy configuration URL field in Firefox).

  1. Why can I connect to the https proxy server through SwitchOmega?

The SwitchOmega extension in fact generates a pac file for Chrome to use, though how it interacts with Chrome is so far unknown to me.

By clicking the Export PAC button in SwitchOmega, I get a file contains:

var FindProxyForURL = function(init, profiles) {
    return function(url, host) {
        "use strict";
        var result = init, scheme = url.substr(0, url.indexOf(":"));
        do {
            result = profiles[result];
            if (typeof result === "function") result = result(url, host, scheme);
        } while (typeof result !== "string" || result.charCodeAt(0) === 43);
        return result;
    };
}("+test", {
    "+test": function(url, host, scheme) {
        "use strict";
        if (/^127\.0\.0\.1$/.test(host) || /^::1$/.test(host) || /^localhost$/.test(host)) return "DIRECT";
        return "HTTPS myHttpsProxyServer.com:9999"; // This line matters
    }
});

From https://developer.mozilla.org/en-US/docs/Web/HTTP/Proxy_servers_and_tunneling/Proxy_Auto-Configuration_(PAC)_file:

HTTP host:port   
The specified proxy should be used   
HTTPS host:port 
The specified HTTPS proxy should be used  
  1. I think I build a https proxy server. But why others are saying that "There's no such thing as a https proxy server?

Yes I build a https proxy server/a http proxy server over tls connection. Those who says "There's no such thing as a https proxy server" are wrong.

Spradling answered 11/7, 2019 at 15:57 Comment(4)
This is an excellent post. I have one more question, is the HTTPS proxy server you built able to proxy HTTP requests from client? If possible, does that mean the communications between client and proxy is encypted while the communications between proxy and HTTP server isn't?Britt
@ZhaoxingLu Yes, the tls/ssl layer is stripped after proxy.Guimond
@ZhaoxingLu 1. is the HTTPS proxy server you built able to proxy HTTP requests from client? Yes, but it is said that the browser would prefer use the HTTP proxy when both http proxy and https tunnel are available. 2. If using a https proxy, the communication between client and proxy is encrypted. I guess with this you can for example hide your dest host from man-in-the-middle or employ some http auth e.g. Baisc auth.Spradling
3. I believe (not tested) the communication between proxy and dest http server only depends on whether the client is visiting via http or https.Spradling
M
0

AFAIK there is no official HTTPS Proxy specification.

What was Rick said about in his answer is Squid's hack to allow them to cache content that cannot be cached before because of https by terminating browser TLS request then initiating another TLS request to intended server. In essence, Squid is acting as an MITM between browser and server. Altough the connection still protected by TLS, Squid can see the traffic. If the web server is implementing HSTS, browser won't allow us to connect.

What i expected from HTTPS Proxy is to browser initiate both TLS connection. It first initiate TLS connection to proxy server, then request CONNECT to intended web server and initiate another TLS.

Mixologist answered 13/1, 2023 at 10:22 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.