HTTPS connections over proxy servers

Question

Is it possible to have HTTPS connections over proxy servers? If yes, what kind of proxy server allows this?

Duplicated with How to use Socks 5 proxy with Apache HTTP Client 4?

Frogmouth

I think it's not duplicated with #22938483 – Stander

Yes, it possible. See practical examples here #56982493 – Carnallite

Answer 1

TLS/SSL (The S in HTTPS) guarantees that there are no eavesdroppers between you and the server you are contacting, i.e. no proxies. Normally, you use CONNECT to open up a TCP connection through the proxy. In this case, the proxy will not be able to cache, read, or modify any requests/responses, and therefore be rather useless.

If you want the proxy to be able to read information, you can take the following approach:

1. Client starts HTTPS session
2. Proxy transparently intercepts the connection and returns an ad-hoc generated(possibly weak) certificate K_a, signed by a certificate authority that is unconditionally trusted by the client.
3. Proxy starts HTTPS session to target
4. Proxy verifies integrity of SSL certificate; displays error if the cert is not valid.
5. Proxy streams content, decrypts it and re-encrypts it with K_a
6. Client displays stuff

An example is Squid's SSL bump. Similarly, burp can be configured to do this. This has also been used in a less-benign context by an Egyptian ISP.

Note that modern websites and browsers can employ HPKP or built-in certificate pins which defeat this approach.

Foliose

This could work in principle, but that's not the way browsers talk to HTTP proxies for HTTPS requests. The way it's described here implies that the proxy server is effectively a Man-In-The-Middle (so would have to be trusted accordingly). – Falsify

Squid does this. It's called SSL Bump. – Heard

Won't work w/o lots of alert to end user. "unconditionally trusted by the client" - there's no such thing. Even is the cert is perfect-AAA+++, it still shows different domain not matching to what end user asked for, which will make any sane browser (not meaning IE here...) jumping up and down screaming. Of course, it's possible to use wget with parameters disabling SSL checks, but guess what? this connection cannot be named "SSL" anymore after it's core security checks being disabled. – Bradleigh

@Van Jone Unconditionally trusted refers to a CA cert. CA certs do not have domains. I have amended the answer with two examples where this does/did work in practice without any alerts to the user. – Foliose

@Foliose We're talking about different things. Yes, bogus CA can certify validity of bogus cert if installed to client browser/OS but this is not the supposed mean to defeat MITM. Anti-MITM case here is the client still knows that perfectly valid (though bogus) cert belongs to different server's domain. Clients validate connections by server certs not by CA, so they have mean to match DNs and alert user about untrusted connection (which user is free to ignore in their ignorance, of course). Squid BUMP page notes about this alert too. P.S. I don't oppose correctness of 1st part of answer to OP. – Bradleigh

My answer relies on what you call a "bogus CA". The certificate of the CA is unconditionally trusted, either because the user(or software on his computer, for example enterprise configuration or malware) configured it that way, or because the CA was obtained from one of the CAs trusted by the major browsers, like in the MCS case. The proxy generates a new valid certificate for every domain the client requests, so without the anti-MITM facilities mentioned at the end of the answer the client will not notice. – Foliose

Regarding 2.) - how does the proxy know for which domain (CN) he has to issue the adhoc-certificate? Within the handshake the domain name is not transmitted yet. – Edgy

@FlorianD.Same way the webserver knows the domain - SNI, supported by all modern browsers. – Foliose

I don't think "have HTTPS connections over proxy servers" means the Man-in-the-Middle attack type of proxy server. I think it's asking whether one can connect to a http proxy server over TLS. And the answer is yes. Check my answer to this question for more details. I was misled by this answer when I was searching for related materials on the net. It's the first result of Google search btw :-). – Carnallite

@Carnallite The question asks about HTTPS connections over proxies. To connect to your proxy, you can of course use any protocol that the proxy itself supports, including TLS. – Foliose

Answer 2

The short answer is: It is possible, and can be done with either a special HTTP proxy or a SOCKS proxy.

First and foremost, HTTPS uses SSL/TLS which by design ensures end-to-end security by establishing a secure communication channel over an insecure one. If the HTTP proxy is able to see the contents, then it's a man-in-the-middle eavesdropper and this defeats the goal of SSL/TLS. So there must be some tricks being played if we want to proxy through a plain HTTP proxy.

The trick is, we turn an HTTP proxy into a TCP proxy with a special command named CONNECT. Not all HTTP proxies support this feature but many do now. The TCP proxy cannot see the HTTP content being transferred in clear text, but that doesn't affect its ability to forward packets back and forth. In this way, client and server can communicate with each other with help of the proxy. This is the secure way of proxying HTTPS data.

There is also an insecure way of doing so, in which the HTTP proxy becomes a man-in-the-middle. It receives the client-initiated connection, and then initiate another connection to the real server. In a well implemented SSL/TLS, the client will be notified that the proxy is not the real server. So the client has to trust the proxy by ignoring the warning for things to work. After that, the proxy simply decrypts data from one connection, reencrypts and feeds it into the other.

Finally, we can certainly proxy HTTPS through a SOCKS proxy, because the SOCKS proxy works at a lower level. You may think a SOCKS proxy as both a TCP and a UDP proxy.

Bewilder

Would using CONNECT cause security warnings as mentioned in https://mcmap.net/q/168224/-convert-http-proxy-to-https-proxy-in-twisted ? – Doorn

@Doorn I don't think so. In CONNECT mode the proxy works at the transport layer. – Bewilder

So, by CONNECT method, any https data from client is not passed to application level of the intermediary proxy? And just evaluated at TCP level of proxy and relayed to remote server directly? – Burdett

@Burdett A TCP CONNECT request is just used to open up a TCP connection through the proxy to the requested webserver. The proxy is only concerned about the transport layer (TCP connection) and does not touch any PDU of higher layers. – Klaus

Answer 3

as far as i can remember, you need to use a HTTP CONNECT query on the proxy. this will convert the request connection to a transparent TCP/IP tunnel.

so you need to know if the proxy server you use support this protocol.

Whiteman

Indeed, clients use the CONNECT verb to use https:// URIs via HTTP proxy servers. In this case, the connection is tunnelled through the proxy, so the certificate verification is done as usual, as if the client was talking directly to the end server. – Falsify

@chburd, But do proxies usually support HTTP CONNECT? – Doorn

Answer 4

If it's still of interest, here is an answer to a similar question: Convert HTTP Proxy to HTTPS Proxy in Twisted

To answer the second part of the question:

If yes, what kind of proxy server allows this?

Out of the box, most proxy servers will be configured to allow HTTPS connections only to port 443, so https URIs with custom ports wouldn't work. This is generally configurable, depending on the proxy server. Squid and TinyProxy support this, for example.

Falsify

Answer 5

Here is my complete Java code that supports both HTTP and HTTPS requests using SOCKS proxy.

import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.Proxy;
import java.net.Socket;
import java.nio.charset.StandardCharsets;

import org.apache.http.HttpHost;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.config.Registry;
import org.apache.http.config.RegistryBuilder;
import org.apache.http.conn.socket.ConnectionSocketFactory;
import org.apache.http.conn.socket.PlainConnectionSocketFactory;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.protocol.HttpContext;
import org.apache.http.ssl.SSLContexts;
import org.apache.http.util.EntityUtils;

import javax.net.ssl.SSLContext;

/**
    * How to send a HTTP or HTTPS request via SOCKS proxy.
    */
public class ClientExecuteSOCKS {

    public static void main(String[] args) throws Exception {
        Registry<ConnectionSocketFactory> reg = RegistryBuilder.<ConnectionSocketFactory>create()
            .register("http", new MyHTTPConnectionSocketFactory())
            .register("https", new MyHTTPSConnectionSocketFactory(SSLContexts.createSystemDefault
                ()))
            .build();
        PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(reg);
        try (CloseableHttpClient httpclient = HttpClients.custom()
            .setConnectionManager(cm)
            .build()) {
            InetSocketAddress socksaddr = new InetSocketAddress("mysockshost", 1234);
            HttpClientContext context = HttpClientContext.create();
            context.setAttribute("socks.address", socksaddr);

            HttpHost target = new HttpHost("www.example.com/", 80, "http");
            HttpGet request = new HttpGet("/");

            System.out.println("Executing request " + request + " to " + target + " via SOCKS " +
                "proxy " + socksaddr);
            try (CloseableHttpResponse response = httpclient.execute(target, request, context)) {
                System.out.println("----------------------------------------");
                System.out.println(response.getStatusLine());
                System.out.println(EntityUtils.toString(response.getEntity(), StandardCharsets
                    .UTF_8));
            }
        }
    }

    static class MyHTTPConnectionSocketFactory extends PlainConnectionSocketFactory {
        @Override
        public Socket createSocket(final HttpContext context) throws IOException {
            InetSocketAddress socksaddr = (InetSocketAddress) context.getAttribute("socks.address");
            Proxy proxy = new Proxy(Proxy.Type.SOCKS, socksaddr);
            return new Socket(proxy);
        }
    }

    static class MyHTTPSConnectionSocketFactory extends SSLConnectionSocketFactory {
        public MyHTTPSConnectionSocketFactory(final SSLContext sslContext) {
            super(sslContext);
        }

        @Override
        public Socket createSocket(final HttpContext context) throws IOException {
            InetSocketAddress socksaddr = (InetSocketAddress) context.getAttribute("socks.address");
            Proxy proxy = new Proxy(Proxy.Type.SOCKS, socksaddr);
            return new Socket(proxy);
        }
    }
}

Flotsam

Answer 6

You can accomplish this using man-in-the-middle techniques with dynamic SSL generation. Take a look at mitmproxy - it's a Python based, SSL-capable MITM proxy.

Bugaboo

Answer 7

Tunneling (port forwarding) HTTPS through SSH (Linux version):

1. Turn off using 443 on localhost.

2. Start tunneling as root:

   ssh -N login@proxy_server -L 443:target_ip:443
   

3. Adding 127.0.0.1 target_domain.com to /etc/hosts.

Everything you do on localhost.
Then: target_domain.com is accessible from localhost browser.

Donatist

Answer 8

I don't think "have HTTPS connections over proxy servers" means the Man-in-the-Middle attack type of proxy server. I think it's asking whether one can connect to a http proxy server over TLS. And the answer is yes.

---

Is it possible to have HTTPS connections over proxy servers?

Yes, see my question and answer here. HTTPs proxy server only works in SwitchOmega

If yes, what kind of proxy server allows this?

The kind of proxy server deploys SSL certificates, like how ordinary websites do. But you need a pac file for the brower to configure proxy connection over SSL.

Carnallite

Answer 9

I had tried

• start tunneling: ssh -N -D 12345 login@proxy_server
• Setting the proxy in the firefox settings as localhost:12345
  • and ticking "use this proxy for all protocols"

but this resulted in the error "Insecure connection" whenever I tried to connect to an https website.

The solution was to

• "untick" the "use this proxy for all protocols"
• set the proxy "localhost:12345" only as a SOCKS proxy
• and leave the HTTP proxy, SSL proxy, FTP proxy blank

Reference from digital ocean documentation

How To Route Web Traffic Securely Without a VPN Using a SOCKS Tunnel

Bringingup