HttpClient: ssl handshake on every request

Question

I use static HttpClient, and it works very slowly over https. I have added -Djavax.net.debug=ssl and found that handshaking is started for every https request again. looks like it can not reuse old session, but I can not found why.

9007199254743735, setSoTimeout(0) called
Allow unsafe renegotiation: false
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
9007199254743735, setSoTimeout(0) called
%% No cached client session
*** ClientHello, SSLv3
...
%% Didn't cache non-resumable client session: [Session-1, SSL_RSA_WITH_RC4_128_MD5]
...
Is initial handshake: true

BTW. before I was faced with another problem on this host: "Received fatal alert: bad_record_mac", it was solved by allowing only SSLv3

UPD1: HttpClient init code

    final SSLContext sslCtx;
    sslCtx = SSLContext.getInstance("SSL");
    sslCtx.init(null, new TrustManager[]{new X509TrustManager() {
            @Override
            public void checkClientTrusted(X509Certificate[] cert,
                    String authType) {
            }

            @Override
            public void checkServerTrusted(X509Certificate[] cert,
                    String authType) {
            }

            @Override
            public X509Certificate[] getAcceptedIssuers() {
                return null;
            }
        }}, null);

    X509HostnameVerifier verifier = new X509HostnameVerifier() {
        @Override
        public void verify(String string, SSLSocket ssls) throws IOException {
        }

        @Override
        public void verify(String string, X509Certificate xc) throws SSLException {
        }

        @Override
        public void verify(String string, String[] strings, String[] strings1) throws SSLException {
        }

        @Override
        public boolean verify(String string, SSLSession ssls) {
            return true;
        }
    };
    final SSLSocketFactory socketFactory = new SSLv3SocketFactory(sslCtx, verifier);
    final SchemeRegistry registry = new SchemeRegistry();
    registry.register(new Scheme("https", 443, socketFactory));

    final PoolingClientConnectionManager cm = new PoolingClientConnectionManager(registry);
    cm.setMaxTotal(100);
    cm.setDefaultMaxPerRoute(50);
    final HttpParams httpParams = new BasicHttpParams();
    HttpConnectionParams.setSoTimeout(httpParams, timeout);

    httpClient = new DefaultHttpClient(cm, httpParams);

    ((DefaultHttpClient) httpClient).setKeepAliveStrategy(new ConnectionKeepAliveStrategy() {
        @Override
        public long getKeepAliveDuration(HttpResponse hr, HttpContext hc) {
            return 0;
        }
    });
    httpClient.getParams().setParameter("http.socket.timeout", 900000);

UPD2: modified SSLSocketFactory("Received fatal alert: bad_record_mac" issue)

  public class SSLv3SocketFactory extends SSLSocketFactory {

    private final javax.net.ssl.SSLSocketFactory socketfactory;

    public SSLv3SocketFactory(SSLContext sslContext, X509HostnameVerifier hostnameVerifier) {
        super(sslContext, hostnameVerifier);
        this.socketfactory = sslContext.getSocketFactory();
    }

    @Override
    public Socket createLayeredSocket(
            final Socket socket,
            final String host,
            final int port,
            final boolean autoClose) throws IOException, UnknownHostException {
        SSLSocket sslSocket = (SSLSocket) this.socketfactory.createSocket(
                socket,
                host,
                port,
                autoClose);
        sslSocket.setEnabledProtocols(new String[]{"SSLv3"});


        return sslSocket;
    }

    @Override
    public Socket connectSocket(
            final Socket socket,
            final InetSocketAddress remoteAddress,
            final InetSocketAddress localAddress,
            final HttpParams params) throws IOException, UnknownHostException, ConnectTimeoutException {

        if (socket instanceof SSLSocket) {
            ((SSLSocket) socket).setEnabledProtocols(new String[]{"SSLv3"});;
        }
        return super.connectSocket(socket, remoteAddress, localAddress, params);
    }
}

UPD3: Problem exists only for SSLv3, TLSv1 works fine

Godding

Please post code of how you do the requests in the http client. – Watercool

I have added HttpClient init code, than I just call httpClient.execute(), url is always same – Godding

Answer 1

HttpClient re-uses persistent SSL connections with client authentication only if it can make sure they belong to the same user / security context (for obvious reasons).

Make sure you are using the same HttpContext for all logically related requests. This will ensure the security principal (DN of the client certificate) will get propagated between individual HTTP requests.

Follow-up

It tuned out the server simply does not want connections to be re-used. Every response contains 'Connection: close' directive that prompts the client to close connections after receiving the response. It may happen, though, that the server treats different clients differently based on the request message composition. Try masquerading HttpClient by using a different User-Agent header value and see if that makes any difference.

Brunildabruning

I have tried to create static HttpContext , and specify it every time I call httpClient.execute(). BTW This problem occurs only with 1 server. – Godding

I'd be surprised if HttpClient or HttpContext had anything to do with it. SSL Session support occurs at the SSLSocket layer, not the HTTP layer. – After

@EJP: Believe me, it does. HttpClient treats SSL connections with client authentication as stateful and relies on the context to propagate that state – Brunildabruning

@John: Please use HttpClient context logging to produce a log of conn management operations as described here hc.apache.org/httpcomponents-client-ga/logging.html – Brunildabruning

@oleg The SSLContext and SSLSession and SSLSessionContext are in the JSSE layer. You seem to be confusing HTTP sessions with SSL sessions. I can see nothing in the Apache source code that supports your contention. – After

@EJP: it just happens so that I know HttpClient inner working quite a bit. I am not talking about SSL session or context but rather about HTTP context (which among many things includes user security principal) that directly affects connection re-use. See hc.apache.org/httpcomponents-client-ga/httpclient/xref/org/… – Brunildabruning

@John: please see follow-up – Brunildabruning

@oleg, yes, I see that server close connection after every request, but when I use C# HttpSimpleClientProtocol server doesn't close it and it works very fast, so I think, that maybe it still can be fixed on client side – Godding

@oleg In that case you aren't answering the question. The question is about SSL session resumption in the SSL handshake, which concerns the SSL session. It has nothing to do with the HTTP session. – After

@EJP: This is about HTTP connection persistence and connection re-use in the first place. SSL handshake on each request is a direct consequence of connections not being re-used or kept alive at the HTTP level. SSL session caching and re-use can help alleviate the problem but only to an extent. – Brunildabruning

@John: As I already said the only explanation I can offer is that the server responds differently to different clients based request composition. – Brunildabruning

@oleg No. The question is about 'SSL handshake on every request'. SSL sessions can extend across TCP connections. They're not affected by HTTP connection persistence or the lack of it. A new TCP connection joining an existing SSL session performs a very abbreviated SSL handshake. – After

Answer 2

As you state in a comment that the problem only occurs with one server, clearly the problem is at that server. They have set a very short SSL session timeout, or disabled session resumption altogether somehow.

There's nothing you can do about it from your end.

After

C# HttpSimpleClientProtocol works fine with this server, so I guess that is possible to "fix" java HttpClient – Godding