How to make Java 6, which fails SSL connection with "SSL peer shut down incorrectly", succeed like Java 7?

Question

I'm seeing an SSL connection from a client running Java 6 fail with an exception like:

Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:882)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1188)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1215)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1199)
    at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:434)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:166)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:133)
    ... 35 more
Caused by: java.io.EOFException: SSL peer shut down incorrectly
    at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:462)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:863)
    ... 41 more

The server is a Tomcat 7-based app, running on Java 7, Linux, and on Amazon EC2, for what that's worth.

I've found a lot of suggestions on possible casues, including connecting accidentally to a non-SSL port, etc. I believe I've ruled it all out, mostly because the exact same client works when running Java 7 with no change. (OS X in both cases.)

Below I include the debug output from Java 6's and Java 7's SSL connection procedure. My question to experts is, does this suggest that some possible cipher or protocol setting, maybe a default in Java 7, could be enabled in Java 6 to make it work?

Java 6:

Allow unsafe renegotiation: false
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
%% No cached client session
*** ClientHello, TLSv1
RandomCookie:  GMT: 1363993281 bytes = { 77, 153, 100, 72, 45, 178, 253, 243, 195, 167, 17, 151, 39, 247, 148, 102, 213, 129, 39, 17, 26, 139, 157, 154, 63, 88, 41, 160 }
Session ID:  {}
Cipher Suites: [SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_RC4_40_MD5, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods:  { 0 }
***
main, WRITE: TLSv1 Handshake, length = 81
main, WRITE: SSLv2 client hello message, length = 110
main, received EOFException: error
main, handling exception: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
main, SEND TLSv1 ALERT:  fatal, description = handshake_failure
main, WRITE: TLSv1 Alert, length = 2
main, called closeSocket()

Java 7:

Ignoring unavailable cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
Ignoring unavailable cipher suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA
Ignoring unavailable cipher suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA
Ignoring unsupported cipher suite: TLS_DHE_DSS_WITH_AES_128_CBC_SHA256
Ignoring unsupported cipher suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256
Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256
Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384
Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA256
Ignoring unavailable cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
Ignoring unavailable cipher suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256
Ignoring unavailable cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA
Ignoring unavailable cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA
Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_128_CBC_SHA256
Allow unsafe renegotiation: false
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
main, setSoTimeout(0) called
%% No cached client session
*** ClientHello, TLSv1
RandomCookie:  GMT: 1363993435 bytes = { 131, 83, 80, 186, 215, 90, 171, 131, 231, 18, 184, 183, 249, 155, 197, 204, 73, 1, 74, 79, 32, 142, 236, 28, 111, 37, 58, 255 }
Session ID:  {}
Cipher Suites: [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_RC4_128_SHA, TLS_ECDH_ECDSA_WITH_RC4_128_SHA, TLS_ECDH_RSA_WITH_RC4_128_SHA, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_RC4_128_MD5, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods:  { 0 }
Extension elliptic_curves, curve names: {secp256r1, sect163k1, sect163r2, secp192r1, secp224r1, sect233k1, sect233r1, sect283k1, sect283r1, secp384r1, sect409k1, sect409r1, secp521r1, sect571k1, sect571r1, secp160k1, secp160r1, secp160r2, sect163r1, secp192k1, sect193r1, sect193r2, secp224k1, sect239k1, secp256k1}
Extension ec_point_formats, formats: [uncompressed]
Extension server_name, server_name: [host_name: ec2-xx-xx-xx-xx.compute-1.amazonaws.com]

Glottology

Is the failure visible from the client? I've seen these types of errors before as part of normal SSL negotiation, i.e. it "worked" from the user's point of view. – Lusty

With Java 6, the client hello falls back to SSLv2 whereas with Java 7 it seems to be conducted with TLSv1. I believe that JSSE under Java 6 does not officially support SSL V2 but I need to research that – Chamois

Using ` openssl s_client -connect servername:443` you can debug what protocol the server supports and perform tests by forcing flags like ssl2, ssl2, check this – Chamois

@Lusty yes the request fails every time in Java 6, and the connection does not succeed. – Glottology

@BGR yes I've confirmed that it only supports tls1 and ssl3, as expected I guess. Now I'm looking at how to make Java 6 not use SSLv2. – Glottology

@SeanOwen - how do we debug this issue and find out the cause ? i.e. arrive at the same solution as you ? – Mystagogue

Answer 1

Bruno's answer was the correct one in the end. This is most easily controlled by the https.protocols system property. This is how you are able to control what the factory method returns. Set to "TLSv1" for example.

Glottology

It leads to problems when you want to use two protocols instead of one. For example, SSLv3 and TLSv1. This -Dhttps.protocols=TLSv1,SSLv3 will lead to exceptions if you try to connect either SSLv3 or TLSv1. – Fahlband

@Fahlband Really? What exceptions? Why? – Stores

Answer 2

It seems that in the debug log for Java 6 the request is send in SSLv2 format.

main, WRITE: SSLv2 client hello message, length = 110

This is not mentioned as enabled by default in Java 7.
Change the client to use SSLv3 and above to avoid such interoperability issues.

Look for differences in JSSE providers in Java 7 and Java 6

Switch

Yep that's good evidence and indeed the SSLv2 appears to be the problem. This sort of leads to my answer here, although @EJP is right that it would be best to find a way to turn off SSLv2 explicitly. – Glottology

Answer 3

Remove "SSLv2ClientHello" from the enabled protocols on the client SSLSocket or HttpsURLConnection.

Stores

I think that's exactly right, though I am not sure how to do it. I'm building an SSLSocketFactory for the platform. SSLContext.createSSLEngine() makes an SSLEngine which has methods to set (not remove, it seems) protocols. But I don't think this helps the SSLSocketFactory. The SSLContext also only lets you ask for a protocol and not disable whatever others that entails (?). If you happen to know how this goes let me know. – Glottology

Get the enabled protocols, remove "SSLv2ClientHello", and set them to what remains. – Stores

I don't believe there's any such method though. At least I can't see it on SSLContext or SSLSocketFactory. SSLEngine is not helpful I believe since the framework will be making its own later. Will keep looking. – Glottology

Any such method as what? What are you talking about? getEnabledCipherSuites() and setEnabledCipherSuites() both exist. You can also set a system property to control this for the default SSLContext. – Stores

Yes @EJP but that is on SSLSocket and I'm making a factory... which probably means wrapping with a custom SSLSocketFactory. Seems like there should be something cleaner. I'll look for this sys property. – Glottology

@SeanOwen, if you're using HttpsURLConnection, the https.protocols system property should help you set the enabled cipher suites. – Snooty

Confirmed that -Dhttps.protocols=TLSv1 makes this work in Java 6. – Glottology

@SeanOwen If you're creating a socket factory, you get the socket, or create it yourself, and you can call these methods, before you return it. – Stores

Answer 4

update the server arguments from -Dhttps.protocols=SSLv3 to -Dhttps.protocols=TLSv1,SSLv3

Hatbox

Answer 5

Do it like this:

SSLSocket socket = (SSLSocket) sslFactory.createSocket(host, port);
socket.setEnabledProtocols(new String[]{"SSLv3", "TLSv1"});

Incus