How to reach an HTTPS site via proxy with Hyper?

Question

The following is an attempt to reach an HTTPS site via proxy:

extern crate hyper;
extern crate hyper_native_tls;

use hyper::net::HttpsConnector;
use hyper::client::{Client, ProxyConfig};
use hyper_native_tls::NativeTlsClient;

fn main() {
    let ssl = NativeTlsClient::new().unwrap();
    let connector = HttpsConnector::new(ssl);

    let client = Client::with_proxy_config(
        ProxyConfig::new(
            "http", "localhost", 3128, connector, ssl
        )
    );

    let response = client.get("https://httpbin.org").send().unwrap();
    println!("{}", response.headers);
}

I get this error:

error[E0277]: the trait bound `hyper_native_tls::TlsStream<hyper::net::HttpStream>: std::fmt::Debug` is not satisfied
  --> src/main.rs:13:9
   |
13 |         ProxyConfig::new(
   |         ^^^^^^^^^^^^^^^^ the trait `std::fmt::Debug` is not implemented for `hyper_native_tls::TlsStream<hyper::net::HttpStream>`
   |
   = note: `hyper_native_tls::TlsStream<hyper::net::HttpStream>` cannot be formatted using `:?`; if it is defined in your crate, add `#[derive(Debug)]` or manually implement it
   = note: required because of the requirements on the impl of `std::fmt::Debug` for `hyper::net::HttpsStream<hyper_native_tls::TlsStream<hyper::net::HttpStream>>`
   = note: required because of the requirements on the impl of `hyper::net::SslClient<hyper::net::HttpsStream<hyper_native_tls::TlsStream<hyper::net::HttpStream>>>` for `hyper_native_tls::NativeTlsClient`
   = note: required by `<hyper::client::ProxyConfig<C, S>>::new`

error[E0277]: the trait bound `hyper_native_tls::TlsStream<hyper::net::HttpStream>: std::fmt::Debug` is not satisfied
  --> src/main.rs:13:9
   |
13 |           ProxyConfig::new(
   |  _________^ starting here...
14 | |             "http", "localhost", 3128, connector, ssl
15 | |         )
   | |_________^ ...ending here: the trait `std::fmt::Debug` is not implemented for `hyper_native_tls::TlsStream<hyper::net::HttpStream>`
   |
   = note: `hyper_native_tls::TlsStream<hyper::net::HttpStream>` cannot be formatted using `:?`; if it is defined in your crate, add `#[derive(Debug)]` or manually implement it
   = note: required because of the requirements on the impl of `std::fmt::Debug` for `hyper::net::HttpsStream<hyper_native_tls::TlsStream<hyper::net::HttpStream>>`
   = note: required because of the requirements on the impl of `hyper::net::SslClient<hyper::net::HttpsStream<hyper_native_tls::TlsStream<hyper::net::HttpStream>>>` for `hyper_native_tls::NativeTlsClient`
   = note: required by `hyper::client::ProxyConfig`

error[E0277]: the trait bound `hyper_native_tls::TlsStream<hyper::net::HttpStream>: std::fmt::Debug` is not satisfied
  --> src/main.rs:12:18
   |
12 |     let client = Client::with_proxy_config(
   |                  ^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `std::fmt::Debug` is not implemented for `hyper_native_tls::TlsStream<hyper::net::HttpStream>`
   |
   = note: `hyper_native_tls::TlsStream<hyper::net::HttpStream>` cannot be formatted using `:?`; if it is defined in your crate, add `#[derive(Debug)]` or manually implement it
   = note: required because of the requirements on the impl of `std::fmt::Debug` for `hyper::net::HttpsStream<hyper_native_tls::TlsStream<hyper::net::HttpStream>>`
   = note: required because of the requirements on the impl of `hyper::net::SslClient<hyper::net::HttpsStream<hyper_native_tls::TlsStream<hyper::net::HttpStream>>>` for `hyper_native_tls::NativeTlsClient`
   = note: required by `hyper::Client::with_proxy_config`

Here are the Cargo dependencies:

[dependencies]
hyper = "0.10"
hyper-native-tls = "0.2"

Things are better using these dependencies:

[dependencies]
hyper = "0.10"
hyper-openssl = "0.2"

And this code:

extern crate hyper;
extern crate hyper_openssl;

use hyper::net::HttpsConnector;
use hyper::client::{Client, ProxyConfig};
use hyper_openssl::OpensslClient as TlsClient;

fn main() {
    let ssl = TlsClient::new().unwrap();
    let connector = HttpsConnector::new(ssl.clone());

    let client = Client::with_proxy_config(
        ProxyConfig::new(
            "http", "localhost", 3128, connector, ssl
        )
    );

    let response = client.get("https://httpbin.org").send().unwrap();
    println!("{:#?}", response);
}

Output:

Response {
    status: Ok,
    headers: Headers { Server: nginx, Date: Thu, 12 Jan 2017 15:05:13 GMT, Content-Type: text/html; charset=utf-8, Content-Length: 12150, Connection: keep-alive, Access-Control-Allow-Origin: *, Access-Control-Allow-Credentials: true, },
    version: Http11,
    url: "https://httpbin.org/",
    status_raw: RawStatus(
        200,
        "OK"
    ),
    message: Http11Message {
        is_proxied: false,
        method: None,
        stream: Wrapper {
            obj: Some(
                Reading(
                    SizedReader(remaining=12150)
                )
            )
        }
    }
}

No build failures there, but it doesn't go via the proxy.

Dross

Aren't all these errors caused by the fact you are trying to print the response in debug mode while it doesn't implement Debug? – Hema

I get the same error even if I don't print anything – Dross

None of the structures on hyper_native_tls seem to impl Debug, so the behaviour seen on that code would be correct. Please make sure that you are not performing any fmt() and update the question with your findings. – Darg

@E_net4: Should probably be an answer. I checked ProxyConfig and Client and neither require Debug, so it seems the OP is shooting himself in the foot... and that the error message isn't great. Or maybe he's not using the latest version of hyper (0.10.0) and thus the docs I'm consulting are not good. – Decibel

@MatthieuM. In a way, that is true. On the other hand, the question is of poor quality and doesn't refer at all to the use of Debug-formatted output in its phrasing. I'd rather let it either be improved or deleted. I also wonder whether there's a suitable duplicate. – Darg

I've updated the Question @Darg – Dross

And the error still states "cannot be formatted using :?"? – Hema

@Hema yeah, same error – Dross

Answer 1

There were some untested conflicts around the crates hyper_native_tls and native_tls.

Currently, there is a restriction on the implementation of SslClient for NativeTlsClient that requires T: Debug (code). The code in the question does not compile because TlsStream does not implement Debug, regardless of its parameter type.

At first one could consider removing the aforementioned constraint. But that triggers a few other errors in hyper_native_tls:

error[E0277]: the trait bound `T: std::fmt::Debug` is not satisfied
   --> src/lib.rs:129:45
    |
129 |             Err(e) => Err(hyper::Error::Ssl(Box::new(e))),
    |                                             ^^^^^^^^^^^ the trait `std::fmt::Debug` is not implemented for `T`
    |
    = help: consider adding a `where T: std::fmt::Debug` bound
    = note: required because of the requirements on the impl of `std::error::Error` for `native_tls::HandshakeError<T>`
    = note: required for the cast to the object type `std::error::Error + std::marker::Sync + std::marker::Send + 'static`

Going down the rabbit hole, we discover that native_tls::HandshakeError holds a parameter type S of the stream that was interrupted (in case of this particular error). This became another problem because the type only implements Debug where S: Debug, and according to the Error trait, error types must always implement Debug.

A fix to this particular issue is to provide Debug to TlsStream:

#[derive(Debug, Clone)]
pub struct TlsStream<S>(Arc<Mutex<native_tls::TlsStream<S>>>);

The first code snippet still won't compile because ssl is being used after moving, and copying is not tolerated here. The second snippet works by cloning the object, which is unfortunately not implemented for NativeTlsClient. We also cannot derive the implementation because native_tls::TlsConnector does not implement Clone either. As far as this rabbit hole went, it should probably end here before this becomes a debugging report.

I am not entirely sure of what can be done here (aside from not using native TLS at all), but my current advice would be filing an issue in hyper_native_tls_client, explaining that it doesn't work with hyper's client proxy (edit: it's done and fixed!).

Darg

github.com/sfackler/hyper-native-tls/commit/… – Dross

@Tshepang Good job. ;) – Darg