SSL Renegotiation with Client Certificate causes Server Buffer Overflow

Question

I've coded a Java client application which connects to an Apache web server over HTTPS using a client certificate and performs an HTTP PUT of a file to the server. It works fine with small files, but crashes with large ones.

The Apache server log shows the following:

...
OpenSSL: Handshake: done
...
Changed client verification type will force renegotiation
...
filling buffer, max size 131072 bytes
...
request body exceeds maximum size (131072) for SSL buffer
could not buffer message body to allow SSL renegotiation to proceed
...    
OpenSSL: I/O error, 5 bytes expected to read on BIO
(104)Connection reset by peer: SSL input filter read failed.
(32)Broken pipe: core_output_filter: writing data to the network
Connection closed to child 20 with standard shutdown

The response on the client is:

java.io.IOException: Server returned HTTP response code: 401 for URL

I'm not familiar with this process so I'm not sure if renegotiation is necessary here or if there is something I can do to prevent it. Or perhaps I can have the client wait until the renegotiation is complete before sending application data? Here is an excerpt of the client code (error handling removed):

        URL url = new URL("my url goes here");
        con = (HttpsURLConnection) url.openConnection();
        con.setSSLSocketFactory(getMyCustomClientCertSocketFactory());
        con.setRequestMethod("PUT");
        con.setDoOutput(true);
        con.connect();
        writer = new OutputStreamWriter(con.getOutputStream());
        writer.write(xml);
        writer.close();

        parseServerResponse(con.getInputStream());

I'm thinking maybe I need to use a lower level API like SSLSocket and leverage the HandshakeCompletedListener?

I'm also wondering if the Apache SSLVerifyDepth directive has anything to do with why a renegotiation is occurring. I've got the directive in a per-directory context (only one upload directory) with value 2 and The Apache manual says this about it:

In per-directory context it forces a SSL renegotation with the reconfigured client verification depth after the HTTP request was read but before the HTTP response is sent.

As requested here is the Java debugging output:

keyStore is : 
keyStore type is : jks
keyStore provider is : 
init keystore
init keymanager of type SunX509
trustStore is: C:\Program Files\Java\jdk1.6.0_35\jre\lib\security\cacerts
trustStore type is : jks
trustStore provider is : 
init truststore
adding as trusted cert:
 ...
trigger seeding of SecureRandom
done seeding SecureRandom
***
found key for : key-alias
chain [0] = [
[
...
]
***
trigger seeding of SecureRandom
done seeding SecureRandom
Allow unsafe renegotiation: false
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
%% No cached client session
*** ClientHello, TLSv1
RandomCookie:  ...
Session ID:  {}
Cipher Suites: [SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_RC4_40_MD5, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods:  { 0 }
***
main, WRITE: TLSv1 Handshake, length = 75
main, WRITE: SSLv2 client hello message, length = 101
main, READ: TLSv1 Handshake, length = 81
*** ServerHello, TLSv1
RandomCookie:  ...
Session ID:  ...
Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA
Compression Method: 0
Extension renegotiation_info, renegotiated_connection: <empty>
***
%% Created:  [Session-1, TLS_RSA_WITH_AES_128_CBC_SHA]
** TLS_RSA_WITH_AES_128_CBC_SHA
main, READ: TLSv1 Handshake, length = 4392
*** Certificate chain
chain [0] = [
[
...
Certificate Extensions: 8
[1]: ObjectId: 1.3.6.1.5.5.7.1.1 Criticality=false
AuthorityInfoAccess [
  [
   accessMethod: ...
   accessLocation: URIName: ...
   accessMethod: ...
   accessLocation: URIName: ...
]

[2]: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
...
]
]
[3]: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:false
  PathLen: undefined
]
[4]: ObjectId: 2.5.29.31 Criticality=false
CRLDistributionPoints [
  [DistributionPoint:
     [URIName: ...
]]
[5]: ObjectId: 2.5.29.32 Criticality=false
CertificatePolicies [
  [CertificatePolicyId: ...
[PolicyQualifierInfo: [
  qualifierID: ...
  qualifier: ...
]]  ]
]
[6]: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
  serverAuth
  clientAuth
]
[7]: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
  DigitalSignature
  Key_Encipherment
]
[8]: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
  DNSName: ...
]
]
  Algorithm: [SHA1withRSA]
  Signature:
...
]
...
***
main, READ: TLSv1 Handshake, length = 4
*** ServerHelloDone
*** ClientKeyExchange, RSA PreMasterSecret, TLSv1
main, WRITE: TLSv1 Handshake, length = 518
SESSION KEYGEN:
PreMaster Secret:
...
CONNECTION KEYGEN:
Client Nonce:
...
Server Nonce:
...
Master Secret:
...
Client MAC write Secret:
...
Server MAC write Secret:
...
Client write key:
...
Server write key:
...
Client write IV:
...
Server write IV:
...
main, WRITE: TLSv1 Change Cipher Spec, length = 1
*** Finished
verify_data:  { 18, 162, 18, 251, 82, 111, 87, 133, 53, 240, 114, 155 }
***
main, WRITE: TLSv1 Handshake, length = 48
main, READ: TLSv1 Change Cipher Spec, length = 1
main, READ: TLSv1 Handshake, length = 48
*** Finished
verify_data:  { 46, 206, 8, 40, 63, 252, 99, 190, 251, 183, 110, 201 }
***
%% Cached client session: [Session-1, TLS_RSA_WITH_AES_128_CBC_SHA]
main, WRITE: TLSv1 Application Data, length = 256
main, WRITE: TLSv1 Application Data, length = 32
main, WRITE: TLSv1 Application Data, length = 16416
main, WRITE: TLSv1 Application Data, length = 16416
...
main, WRITE: TLSv1 Application Data, length = 16416
main, WRITE: TLSv1 Application Data, length = 16416
main, WRITE: TLSv1 Application Data, length = 512
main, READ: TLSv1 Application Data, length = 304 

As requested here is the getMyCustomClientCertSocketFactory source (obtains certificate and key from a PEM file):

public static SSLSocketFactory getMyCustomClientCertSocketFactory(String pemPath,
        boolean verifyPeer)
        throws NoSuchAlgorithmException, FileNotFoundException, IOException,
        KeyStoreException, CertificateException, UnrecoverableKeyException,
        KeyManagementException, InvalidKeySpecException {
    SSLContext context = SSLContext.getInstance("TLS");

    byte[] certAndKey = IOUtil.fileToBytes(new File(pemPath));
    byte[] certBytes = parseDERFromPEM(certAndKey,
            "-----BEGIN CERTIFICATE-----", "-----END CERTIFICATE-----");
    byte[] keyBytes = parseDERFromPEM(certAndKey,
            "-----BEGIN PRIVATE KEY-----", "-----END PRIVATE KEY-----");

    X509Certificate cert = generateX509CertificateFromDER(certBytes);
    RSAPrivateKey key = generateRSAPrivateKeyFromDER(keyBytes);

    KeyStore keystore = KeyStore.getInstance("JKS");
    keystore.load(null);
    keystore.setCertificateEntry("cert-alias", cert);
    keystore.setKeyEntry("key-alias", key, "changeit".toCharArray(),
            new Certificate[]{cert});

    KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
    kmf.init(keystore, "changeit".toCharArray());

    KeyManager[] km = kmf.getKeyManagers();

    TrustManager[] tm = null;

    if (!verifyPeer) {
        tm = new TrustManager[]{new TrustyTrustManager()};
    }

    context.init(km, tm, null);

    return context.getSocketFactory();
}

Jennings

If I use the UNIX utility 'curl' I can transfer large files without incident so I am wondering what it is doing differently... – Jennings

Here is a bug report which discusses the issue: bugzilla.redhat.com/show_bug.cgi?id=491763. Issuing a HTTP GET (or HEAD or OPTIONS) on the upload directory before the PUT doesn't seem to work when using HttpsUrlConnections - I guess keep-alive isn't being honored? – Jennings

Run your client with -Djavax.net.debug=ssl,handshake and post the output in your answer. – Ieper

The debug output is too verbose to paste in a comment, but basically it looks like the application data gets sent before the renegotiation is even requested by the server and by that time it is too late because the socket gets closed. Is there a way to reliably anticipate a renegotiation? What does curl do? – Jennings

Edit it into your question. Until you do so nobody can assist. – Ieper

This problem, as you have correctly identified, is at the Apache end. Changing things at the Java end won't have any effect. – Ieper

Keep in mind that when I upload a huge file to the client certificate protected directory using the UNIX utility curl it works great. Therefore there IS something that the client can do to prevent flooding the server and my Java client isn't doing it. – Jennings

Answer 1

It would seem that the HttpsUrlConnection facility built into Sun Java cannot handle the large HTTP PUT with client certificate scenario in a server friendly way (i.e. without overflowing the servers SSL renegotiate buffer).

I examined what curl was doing to see what "server friendly meant", and it turns out there is an HTTP 1.1 header named "Expect", which curl sends with value "100-continue" (see spec http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.20). This header essentially says "I've got a huge payload, but before I send it please let me know if you can handle it". This gives the endpoints time to renegotiate the client certificate before the payload is sent.

In the Sun HttpUrlConnection implementation it seems this header is not allowed, and is actually in the restricted headers list; meaning even if you set it with the HttpUrlConnection.setRequestProperty method the header is not actually sent to the server. You can override the restricted headers with the system property sun.net.http.allowRestrictedHeaders, but then the client just crashes with a socket exception since the Sun implementation doesn't know how to handle this part of the protocol.

Interestingly it seems that the OpenJDK implementation of Java does support this header. Also, the Apache HTTP Client library supports this header (http://hc.apache.org/); I've implemented a test program with the Apache HTTP client library and it can successfully perform and HTTP PUT request of a large file using a client certificate and the Expect header.

To recap, the solutions are:

1. Set the Apache SSLRenegBufferSize directive to a huge number (like 64MB). The default is 128K. This solution may create a denial of service risk
2. Configure a host that always requires client certificates, as opposed to one in which only a few directories require it. This will avoid renegotiation. This isn't a good option in my scenario because the majority of users are anonymous or username/password authenticated. There is only a single upload directory for programmatic upload of files. We would have to create a new virtual host with its own SSL certificate just for this one directory.
3. Use a client which supports the HTTP 1.1 Expect header. Unfortunately the Sun Java does not support this out of the box. Must use third party such as Apache HTTP Component Client library or roll your own solution using Java socket API.
4. Leverage HTTP 1.1 persistent connections (pipelining with keep-alive) by initially issuing an HTTP request that doesn't have a big payload, but causes the renegotiation to occur, then reuse the connection for the HTTP PUT. In theory the client should be able to issue an HTTP HEAD or OPTIONS on the upload directory and then reuse the same connection to do the PUT. In order for this to work the persistent connection pool would probably need to only contain one connection to avoid "priming" one connection and then being issued another for the PUT. However, it doesn't seem like the HttpUrlConnection class will keep/reuse persistent connections involving client certificates or SSL because I've been unable to get this solution to work. See (HttpsUrlConnection and keep-alive).

Jennings

Looks like Java 7 actually supports the Expect header. In my testing I got it working in conjunction with ChunkedStreamingMode. So there are two new lines of code: con.setChunkedStreamingMode(0); and con.setRequestProperty("Expect", "100-Continue"); – Jennings

Answer 2

java.io.IOException: Server returned HTTP response code: 401 for URL

This is an application error. It is not caused by SSL layer. I am not sure why you are getting 401 Unauthorized for larger files but you also omit what is getMyCustomClientCertSocketFactory()
Also did you try another method e.g. POST? Did you have same problem?

Assort

Yup, Apache sends a 401 when its buffer overflows during a renegotiation. Using GET, POST, PUT, etc doesn't matter as long as the request is large - the problem is the server buffer is overflowing if the application data is larger than the buffer can hold. And yes, I can bump the buffer up and the problem goes away, but this is a vulnerability for a denial of service and not ideal. I didn't supply source for getMyCustomClientCertSocketFactory because it is only tangentially related - in order to use a client certificate with HttpsUrlConnection you must set your own SocketFactory. – Jennings

I can bump the buffer up and the problem goes away, but this is a vulnerability for a denial of service and not ideal So you are saying that the problem is the buffer size of Apache?Then you should increase it otherwise it is vulnerable to DoS.I mean even if you make some change from the client side, the issue will still be there in the server if DoS is your concern – Assort

I need to support file uploads at least as large as 64MB. The buffer in Apache is a fairly DoS resistant 128KB by default. If I bump it to 64MB then bringing my server down will be very easy (and might be done accidentally by normal users). Again, the REAL issue is that the Java client isn't smart enough to prevent flooding the server whereas some clients (such as curl) are smarter and don't flood the server during a renegotiation. – Jennings

No, it is not an application error, it is an authentication error sent by Apache. – Ieper

@EJP:But HTTP 401 is an HTTP error and HTTP runs on top of SSL.How is then possible that an HTTP response is send as a result of a handshake problem?If you could write an answer to explain this I would upvote it and I am sure others would find it useful. – Assort

I'll turn that around. How could the application send a 401 after a handshake failure? Answer: it can't. Nobody can. Ergo there is no handshake failure. What there is is a failure prior to renegotiation, in OpenSSL, so no handshake, so no client certificate, so no authentication, so Apache sends 401. – Ieper

@EJP:You mean OpenSSL throws some kind of exception that makes Apache send a 401? – Assort

@Assort Don't put words into my mouth. I meant what I said: no more, no less. I don't know what OpenSSL does, but I do know that if Apache wants a client certificate to authenticate access to the resource and it doesn't get one it will return a 401, and I also know that there is no way anybody can deliver a recognizable 401 after a handshake failure. – Ieper

@EJP:If Apache wants a client certificate for authentication, it will request it as part of the SSL handshake.If it does not get one and the client authentication is mandatory, the connection will be rejected by the server.The only explanation I can think consistent with your comments, is that the handshake is succesfull and Apache does not reject the connection due to missing certificate during handshake but later with a 401.Strange behavior.Tomcat does not work like that – Assort

@Assort Nothing strange about it. The client certificate is not required to be sent in response to a CertificateRequest. The handshake still succeeds. This is defined by neither Apache, OpenSSL, Tomcat, JSSE, etc., but by RFC 2246. OpenSSL, Apache, JSSE, and therefore Tomcat do indeed work like that. In JSSE terms, the (re-)handshake succeeds but a subsequent getPeerCertificate() call by the server throws PeerUnverifiedException. Similar things happen in OpenSSL and therefore within Apache. At this point Tomcat, Apache, etc. can all send a 401 over the still-existing HTTPS connection. – Ieper

@EJP:I see.Thanks for explaining this! – Assort

Answer 3

Based on all the extra information now supplied, I think you should write the XML in multiple chunks rather than just one. At present you are writing one chunk, which will be chunked by SSL into 16k chunks, which is choking Apache for some reason (it shouldn't). I would try chunk sizes no bigger than 4k. Tune the chunk size until it works.

You may well find client certificate problems once you get past this one. Don't be discouraged, it's proof that you have at least solved this problem.

Ieper

I don't understand this answer. How did you come to the conclusion that Apache is choking on 16k chunks? If that is the problem then how do I change the chunk size? – Jennings