Login/Register
TLS handshake fails between a Java 1.8 client and a Java 1.7 TLS 1.1 server running in FIPS mode, even after disabling TLS 1.2 in the client side
Asked Answered
Solved securityssljava-8fipsnss
S

1

7

The SSL / TLS handshake between a "Java 1.7 TLS 1.1 server" and a "Java 1.8 client" fails in my environment with the following exception on the server side:

java.security.NoSuchAlgorithmException: no such algorithm: SunTls12MasterSecret for provider SunPKCS11-NSSfips

Following are the details of the server and the client in my environment:

Server:

  • The server uses Java 1.7u45 and is running in FIPS 140 compliant mode as mentioned in http://docs.oracle.com/javase/7/docs/technotes/guides/security/jsse/FIPS.html.
  • The server uses Mozilla Network Security Services (NSS) version 3.14.3 as the cryptographic provider.
  • The server is running on SLES 11 SP3 (SuSE Linux Enterprise Server) operating system and uses the following NSS RPMS provided by the OS:
    • mozilla-nss-3.14.3-0.11.11
    • mozilla-nss-tools-3.14.3-0.11.11
    • mozilla-nspr-4.9.6-0.3.1
    • libfreebl3-3.14.3-0.11.11
    • libsoftokn3-3.14.3-0.11.11
  • An important point to note is that NSS version 3.14.3 DOES NOT support TLS 1.2.
  • The server uses Java 1.7u45 because of the Java bug "8028192 Use of PKCS11-NSS provider in FIPS mode broken" introduced in Java 1.7u51. See "Known Issues" section "Area: security-libs/javax.net.ssl" in http://www.oracle.com/technetwork/java/javase/7u51-relnotes-2085002.html for more details.

Client:

  • The client is a Java Swing based client application which is launched using Java Web Start.
  • The client uses Java 1.8u25 and is running on Windows 7 Professional SP1 operating system.
  • Since the server does not support TLS 1.2, TLS 1.2 has been disabled in the client side in the "Advanced" tab of the Java Control Panel as mentioned in https://docs.oracle.com/javase/8/docs/technotes/guides/deploy/jcp.html#A1153011.

Question:

  • Since the server does not support TLS 1.2, I disabled TLS 1.2 on the client side.
  • I could not figure out why the handshake fails "even after" disabling TLS 1.2 on the client side. Can you please help me with this?

Additional observations:

  • The same scenario works fine if Java 1.7 (any version) is used and TLS 1.2 is disabled on the client side.
  • The same exception "java.security.NoSuchAlgorithmException: no such algorithm: SunTls12MasterSecret for provider SunPKCS11-NSSfips" is thrown in the server side if Java 1.7 (any version) is used and TLS 1.2 is enabled in the client side.
  • It fails if Java 1.8 (any version) is used in the client side, even after disabling TLS 1.2.

Server side SSL debug logs:

  • It seems that the handshake is first initiated with TLS 1.1 using the cipher TLS_RSA_WITH_AES_128_CBC_SHA.
  • After this, for some reason, the handshake is re-initiated with TLS 1.2 using the cipher TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 which fails with the exception "java.security.NoSuchAlgorithmException: no such algorithm: SunTls12MasterSecret for provider SunPKCS11-NSSfips".

TLS 1.1 Java 1.7u45 server + Java 1.8u25 TLS 1.2 disabled client - fails:

2014/11/28 15:03:38 | INFO   | jvm 1    | *** ClientHello, TLSv1.1
2014/11/28 15:03:38 | INFO   | jvm 1    | RandomCookie:  GMT: 1417167224 bytes = { 55, 212, 126, 68, 49, 1, 205, 58, 112, 15, 1, 9, 38, 31, 58, 188, 229, 115, 10, 61, 249, 209, 98, 140, 149, 113, 149, 231 }
2014/11/28 15:03:38 | INFO   | jvm 1    | Session ID:  {84, 120, 65, 114, 3, 180, 96, 53, 232, 47, 28, 70, 58, 150, 117, 9, 169, 7, 94, 233, 94, 198, 136, 202, 240, 130, 18, 23, 89, 10, 220, 111}
2014/11/28 15:03:38 | INFO   | jvm 1    | Cipher Suites: [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_anon_WITH_AES_128_CBC_SHA, TLS_ECDH_anon_WITH_3DES_EDE_CBC_SHA]
2014/11/28 15:03:38 | INFO   | jvm 1    | Compression Methods:  { 0 }
2014/11/28 15:03:38 | INFO   | jvm 1    | Extension elliptic_curves, curve names: {secp256r1, sect163k1, sect163r2, secp192r1, secp224r1, sect233k1, sect233r1, sect283k1, sect283r1, secp384r1, sect409k1, sect409r1, secp521r1, sect571k1, sect571r1, secp160k1, secp160r1, secp160r2, sect163r1, secp192k1, sect193r1, sect193r2, secp224k1, sect239k1, secp256k1}
2014/11/28 15:03:38 | INFO   | jvm 1    | Extension ec_point_formats, formats: [uncompressed]
2014/11/28 15:03:38 | INFO   | jvm 1    | Extension renegotiation_info, renegotiated_connection: <empty>
2014/11/28 15:03:38 | INFO   | jvm 1    | ***
2014/11/28 15:03:38 | INFO   | jvm 1    | %% Resuming [Session-135, TLS_RSA_WITH_AES_128_CBC_SHA]
2014/11/28 15:03:38 | INFO   | jvm 1    | *** ServerHello, TLSv1.1
2014/11/28 15:03:38 | INFO   | jvm 1    | RandomCookie:  GMT: 1417167218 bytes = { 175, 0, 103, 107, 78, 20, 172, 204, 4, 196, 148, 153, 126, 87, 188, 255, 85, 219, 140, 39, 41, 136, 51, 33, 169, 31, 36, 150 }
2014/11/28 15:03:38 | INFO   | jvm 1    | Session ID:  {84, 120, 65, 114, 3, 180, 96, 53, 232, 47, 28, 70, 58, 150, 117, 9, 169, 7, 94, 233, 94, 198, 136, 202, 240, 130, 18, 23, 89, 10, 220, 111}
2014/11/28 15:03:38 | INFO   | jvm 1    | Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA
2014/11/28 15:03:38 | INFO   | jvm 1    | Compression Method: 0
2014/11/28 15:03:38 | INFO   | jvm 1    | Extension renegotiation_info, renegotiated_connection: <empty>
2014/11/28 15:03:38 | INFO   | jvm 1    | ***
2014/11/28 15:03:38 | INFO   | jvm 1    | Cipher suite:  TLS_RSA_WITH_AES_128_CBC_SHA
2014/11/28 15:03:38 | INFO   | jvm 1    | CONNECTION KEYGEN:
2014/11/28 15:03:38 | INFO   | jvm 1    | Client Nonce:
2014/11/28 15:03:38 | INFO   | jvm 1    | 0000: 54 78 41 78 37 D4 7E 44   31 01 CD 3A 70 0F 01 09  TxAx7..D1..:p...
2014/11/28 15:03:38 | INFO   | jvm 1    | 0010: 26 1F 3A BC E5 73 0A 3D   F9 D1 62 8C 95 71 95 E7  &.:..s.=..b..q..
2014/11/28 15:03:38 | INFO   | jvm 1    | Server Nonce:
2014/11/28 15:03:38 | INFO   | jvm 1    | 0000: 54 78 41 72 AF 00 67 6B   4E 14 AC CC 04 C4 94 99  TxAr..gkN.......
2014/11/28 15:03:38 | INFO   | jvm 1    | 0010: 7E 57 BC FF 55 DB 8C 27   29 88 33 21 A9 1F 24 96  .W..U..').3!..$.
2014/11/28 15:03:38 | INFO   | jvm 1    | Master Secret:
2014/11/28 15:03:38 | INFO   | jvm 1    | (key bytes not available)
2014/11/28 15:03:38 | INFO   | jvm 1    | Client MAC write Secret:
2014/11/28 15:03:38 | INFO   | jvm 1    | (key bytes not available)
2014/11/28 15:03:38 | INFO   | jvm 1    | Server MAC write Secret:
2014/11/28 15:03:38 | INFO   | jvm 1    | (key bytes not available)
2014/11/28 15:03:38 | INFO   | jvm 1    | Client write key:
2014/11/28 15:03:38 | INFO   | jvm 1    | (key bytes not available)
2014/11/28 15:03:38 | INFO   | jvm 1    | Server write key:
2014/11/28 15:03:38 | INFO   | jvm 1    | (key bytes not available)
2014/11/28 15:03:38 | INFO   | jvm 1    | ... no IV derived for this protocol
2014/11/28 15:03:38 | INFO   | jvm 1    | qtp1981883520-300, WRITE: TLSv1.1 Handshake, length = 81
2014/11/28 15:03:38 | INFO   | jvm 1    | qtp1981883520-300, WRITE: TLSv1.1 Change Cipher Spec, length = 1
2014/11/28 15:03:38 | INFO   | jvm 1    | *** Finished
2014/11/28 15:03:38 | INFO   | jvm 1    | verify_data:  { 205, 73, 239, 162, 189, 111, 93, 112, 252, 191, 178, 72 }
2014/11/28 15:03:38 | INFO   | jvm 1    | ***
2014/11/28 15:03:38 | INFO   | jvm 1    | qtp1981883520-300, WRITE: TLSv1.1 Handshake, length = 64
2014/11/28 15:03:38 | INFO   | jvm 1    | qtp1981883520-299, READ: TLSv1.1 Change Cipher Spec, length = 1
2014/11/28 15:03:38 | INFO   | jvm 1    | qtp1981883520-299, READ: TLSv1.1 Handshake, length = 64
2014/11/28 15:03:38 | INFO   | jvm 1    | *** Finished
2014/11/28 15:03:38 | INFO   | jvm 1    | verify_data:  { 19, 183, 83, 202, 63, 74, 163, 0, 247, 151, 206, 20 }
2014/11/28 15:03:38 | INFO   | jvm 1    | ***
2014/11/28 15:03:38 | INFO   | jvm 1    | qtp1981883520-299 - /AuthServices/auth/tokens, WRITE: TLSv1.1 Application Data, length = 184
2014/11/28 15:03:38 | INFO   | jvm 1    | qtp1981883520-299 - /AuthServices/auth/tokens, WRITE: TLSv1.1 Application Data, length = 2987
2014/11/28 15:03:38 | INFO   | jvm 1    | qtp1981883520-299, WRITE: TLSv1.1 Application Data, length = 5
2014/11/28 15:03:38 | INFO   | jvm 1    | Allow unsafe renegotiation: false
2014/11/28 15:03:38 | INFO   | jvm 1    | Allow legacy hello messages: true
2014/11/28 15:03:38 | INFO   | jvm 1    | Is initial handshake: true
2014/11/28 15:03:38 | INFO   | jvm 1    | Is secure renegotiation: false
2014/11/28 15:03:38 | INFO   | jvm 1    | Is secure renegotiation: false
2014/11/28 15:03:43 | INFO   | jvm 1    | Thread-31, READ: TLSv1.2 Handshake, length = 207
2014/11/28 15:03:43 | INFO   | jvm 1    | *** ClientHello, TLSv1.2
2014/11/28 15:03:43 | INFO   | jvm 1    | RandomCookie:  GMT: 1417167229 bytes = { 209, 207, 128, 77, 244, 126, 201, 133, 122, 149, 46, 174, 146, 131, 232, 171, 236, 114, 188, 239, 89, 136, 179, 55, 42, 35, 10, 208 }
2014/11/28 15:03:43 | INFO   | jvm 1    | Session ID:  {}
2014/11/28 15:03:43 | INFO   | jvm 1    | Session ID:  {}
2014/11/28 15:03:43 | INFO   | jvm 1    | Cipher Suites: [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_RC4_128_SHA, TLS_ECDH_ECDSA_WITH_RC4_128_SHA, TLS_ECDH_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_RC4_128_MD5, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
2014/11/28 15:03:43 | INFO   | jvm 1    | Compression Methods:  { 0 }
2014/11/28 15:03:43 | INFO   | jvm 1    | Compression Methods:  { 0 }
2014/11/28 15:03:43 | INFO   | jvm 1    | Extension elliptic_curves, curve names: {secp256r1, sect163k1, sect163r2, secp192r1, secp224r1, sect233k1, sect233r1, sect283k1, sect283r1, secp384r1, sect409k1, sect409r1, secp521r1, sect571k1, sect571r1, secp160k1, secp160r1, secp160r2, sect163r1, secp192k1, sect193r1, sect193r2, secp224k1, sect239k1, secp256k1}
2014/11/28 15:03:43 | INFO   | jvm 1    | Extension ec_point_formats, formats: [uncompressed]
2014/11/28 15:03:43 | INFO   | jvm 1    | Extension ec_point_formats, formats: [uncompressed]
2014/11/28 15:03:43 | INFO   | jvm 1    | Extension signature_algorithms, signature_algorithms: SHA512withECDSA, SHA512withRSA, SHA384withECDSA, SHA384withRSA, SHA256withECDSA, SHA256withRSA, SHA224withECDSA, SHA224withRSA, SHA1withECDSA, SHA1withRSA, SHA1withDSA, MD5withRSA
2014/11/28 15:03:43 | INFO   | jvm 1    | ***
2014/11/28 15:03:43 | INFO   | jvm 1    | %% Initialized:  [Session-136, SSL_NULL_WITH_NULL_NULL]
2014/11/28 15:03:43 | INFO   | jvm 1    | %% Initialized:  [Session-136, SSL_NULL_WITH_NULL_NULL]
2014/11/28 15:03:43 | INFO   | jvm 1    | %% Negotiating:  [Session-136, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256]
2014/11/28 15:03:43 | INFO   | jvm 1    | *** ServerHello, TLSv1.2
2014/11/28 15:03:43 | INFO   | jvm 1    | RandomCookie:  GMT: 1417167223 bytes = { 117, 144, 129, 63, 132, 34, 26, 83, 118, 25, 122, 135, 116, 24, 242, 213, 196, 31, 25, 127, 155, 153, 6, 132, 244, 45, 21, 235 }
2014/11/28 15:03:43 | INFO   | jvm 1    | RandomCookie:  GMT: 1417167223 bytes = { 117, 144, 129, 63, 132, 34, 26, 83, 118, 25, 122, 135, 116, 24, 242, 213, 196, 31, 25, 127, 155, 153, 6, 132, 244, 45, 21, 235 }
2014/11/28 15:03:43 | INFO   | jvm 1    | Session ID:  {84, 120, 65, 119, 78, 220, 0, 216, 29, 255, 202, 86, 198, 210, 97, 121, 235, 184, 87, 232, 34, 43, 85, 29, 148, 43, 201, 241, 189, 70, 130, 185}
2014/11/28 15:03:43 | INFO   | jvm 1    | Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
2014/11/28 15:03:43 | INFO   | jvm 1    | Compression Method: 0
2014/11/28 15:03:43 | INFO   | jvm 1    | Extension renegotiation_info, renegotiated_connection: <empty>
2014/11/28 15:03:43 | INFO   | jvm 1    | ***
2014/11/28 15:03:43 | INFO   | jvm 1    | Cipher suite:  TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
2014/11/28 15:03:43 | INFO   | jvm 1    | *** Certificate chain
2014/11/28 15:03:43 | INFO   | jvm 1    | chain [0] = [
2014/11/28 15:03:43 | INFO   | jvm 1    | [
2014/11/28 15:03:43 | INFO   | jvm 1    |   Version: V3
2014/11/28 15:03:43 | INFO   | jvm 1    |   Subject: CN=dev-05.labs.blr.com, O=webserver
2014/11/28 15:03:43 | INFO   | jvm 1    |   Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11
2014/11/28 15:03:43 | INFO   | jvm 1    |
2014/11/28 15:03:43 | INFO   | jvm 1    |   Key:  SunPKCS11-NSSfips RSA public key, 2048 bits (id 1, session object)
2014/11/28 15:03:43 | INFO   | jvm 1    |   Key:  SunPKCS11-NSSfips RSA public key, 2048 bits (id 1, session object)
2014/11/28 15:03:43 | INFO   | jvm 1    |   modulus: 19751124565775544542661601941034719218747286997557229376272410409764009924174952830102822006739940996361158891315994655677031683410457285645708620145915789088144941408425439122384306771006790672852952487887077643219829713631271285091822690455402307000211724434432943370113476924425722411995320247744734057517566666508974254720742261526685687656494544221796453195966155694205640019924093341684193258103280171653517687458035087335731929833587535142452049552301009807817546366586239918288540321429443922231821575519420587811789981092934767950075857907111279056051594689275813767976468618202672668356345198890748632149983
2014/11/28 15:03:43 | INFO   | jvm 1    |   public exponent: 65537
2014/11/28 15:03:43 | INFO   | jvm 1    |   public exponent: 65537
2014/11/28 15:03:43 | INFO   | jvm 1    |   Validity: [From: Tue Nov 25 14:37:52 IST 2014,
2014/11/28 15:03:43 | INFO   | jvm 1    |                To: Thu Nov 01 14:37:52 IST 2114]
2014/11/28 15:03:43 | INFO   | jvm 1    |   Issuer: CN=dev-05.labs.blr.com, O=webserver
2014/11/28 15:03:43 | INFO   | jvm 1    |   SerialNumber: [    0d54f951]
2014/11/28 15:03:43 | INFO   | jvm 1    |
2014/11/28 15:03:43 | INFO   | jvm 1    | Certificate Extensions: 1
2014/11/28 15:03:43 | INFO   | jvm 1    | [1]: ObjectId: 2.5.29.14 Criticality=false
2014/11/28 15:03:43 | INFO   | jvm 1    | SubjectKeyIdentifier [
2014/11/28 15:03:43 | INFO   | jvm 1    | KeyIdentifier [
2014/11/28 15:03:43 | INFO   | jvm 1    | 0000: 94 2D EB EF E8 04 5F 84   B2 BA F6 A5 C9 58 D3 79  .-...._......X.y
2014/11/28 15:03:43 | INFO   | jvm 1    | 0010: 44 8F 40 07                                        D.@.
2014/11/28 15:03:43 | INFO   | jvm 1    | ]
2014/11/28 15:03:43 | INFO   | jvm 1    | ]
2014/11/28 15:03:43 | INFO   | jvm 1    |
2014/11/28 15:03:43 | INFO   | jvm 1    | ]
2014/11/28 15:03:43 | INFO   | jvm 1    |   Algorithm: [SHA256withRSA]
2014/11/28 15:03:43 | INFO   | jvm 1    |   Signature:
2014/11/28 15:03:43 | INFO   | jvm 1    | 0000: 39 59 42 B8 26 F6 64 7E   CA C1 33 7C 60 6A FC 80  9YB.&.d...3.`j..
2014/11/28 15:03:43 | INFO   | jvm 1    | 0010: 5F AF 51 89 98 B7 AC 0C   27 DA A1 60 AD 5B 87 11  _.Q.....'..`.[..
2014/11/28 15:03:43 | INFO   | jvm 1    | 0020: D8 95 E3 37 D2 CB E3 8A   6F CF 82 F3 4C AA B6 42  ...7....o...L..B
2014/11/28 15:03:43 | INFO   | jvm 1    | 0030: F5 8B 67 0B D9 F2 3E FA   FE 81 C5 77 78 47 E2 61  ..g...>....wxG.a
2014/11/28 15:03:43 | INFO   | jvm 1    | 0040: 33 DC 97 CB FC 04 1D 99   18 84 C3 DC 28 8D 14 D7  3...........(...
2014/11/28 15:03:43 | INFO   | jvm 1    | 0050: AF 71 1C E6 41 FC D1 71   CB C3 50 66 5E 28 AF EB  .q..A..q..Pf^(..
2014/11/28 15:03:43 | INFO   | jvm 1    | 0060: AF 80 52 CC 89 BE 0D 0B   58 1C CA 1C 34 36 BA 96  ..R.....X...46..
2014/11/28 15:03:43 | INFO   | jvm 1    | 0070: F2 FE 18 73 6B F7 09 35   94 AC 8E CB F2 83 47 62  ...sk..5......Gb
2014/11/28 15:03:43 | INFO   | jvm 1    | 0080: 20 FD 64 64 72 D6 89 D7   77 A7 D0 17 43 7E FF 44   .ddr...w...C..D
2014/11/28 15:03:43 | INFO   | jvm 1    | 0090: 57 B5 1D 27 24 1D F5 87   86 E9 29 EF DE E7 D2 2E  W..'$.....).....
2014/11/28 15:03:43 | INFO   | jvm 1    | 00A0: 32 EE 3D 82 7C 53 7E 93   E9 5F 5E 9C 62 F5 31 C7  2.=..S..._^.b.1.
2014/11/28 15:03:43 | INFO   | jvm 1    | 00B0: 9E 54 58 50 01 EE 58 18   81 6D 52 C0 EB CA CA 52  .TXP..X..mR....R
2014/11/28 15:03:43 | INFO   | jvm 1    | 00C0: 26 CC 3C 9D E5 60 BE BE   A1 E6 D5 79 66 F9 0C FD  &.<..`.....yf...
2014/11/28 15:03:43 | INFO   | jvm 1    | 00D0: BB 9E 36 E5 31 FC D4 68   8A 06 8D A0 0B 68 BA 2B  ..6.1..h.....h.+
2014/11/28 15:03:43 | INFO   | jvm 1    | 00E0: 1E AF 51 4A 6C BC 2D 7D   B4 04 EA D6 DA 28 9B 64  ..QJl.-......(.d
2014/11/28 15:03:43 | INFO   | jvm 1    | 00F0: F9 FF 35 7B E7 91 02 01   37 E7 C2 AA 8D 1E 48 22  ..5.....7.....H"
2014/11/28 15:03:43 | INFO   | jvm 1    |
2014/11/28 15:03:43 | INFO   | jvm 1    | ]
2014/11/28 15:03:43 | INFO   | jvm 1    | ***
2014/11/28 15:03:43 | INFO   | jvm 1    | *** ECDH ServerKeyExchange
2014/11/28 15:03:43 | INFO   | jvm 1    | Signature Algorithm SHA512withRSA
2014/11/28 15:03:43 | INFO   | jvm 1    | Server key: SunPKCS11-NSSfips EC public key, 256 bits (id 1668, session object)
2014/11/28 15:03:43 | INFO   | jvm 1    |   public x coord: 22811020849167726801730368600918463139597169803826118722525163464343792847845
2014/11/28 15:03:43 | INFO   | jvm 1    |   public y coord: 73886304187565809239631250457098470068449769526968865962213829575389354072377
2014/11/28 15:03:43 | INFO   | jvm 1    |   parameters: secp256r1 [NIST P-256, X9.62 prime256v1] (1.2.840.10045.3.1.7)
2014/11/28 15:03:43 | INFO   | jvm 1    | *** ServerHelloDone
2014/11/28 15:03:43 | INFO   | jvm 1    | Thread-31, WRITE: TLSv1.2 Handshake, length = 1237
2014/11/28 15:03:43 | INFO   | jvm 1    | Thread-31, READ: TLSv1.2 Handshake, length = 70
2014/11/28 15:03:43 | INFO   | jvm 1    | *** ECDHClientKeyExchange
2014/11/28 15:03:43 | INFO   | jvm 1    | ECDH Public value:  { 4, 121, 116, 89, 85, 251, 91, 15, 91, 227, 244, 77, 243, 1, 197, 145, 33, 117, 182, 143, 76, 42, 19, 121, 131, 88, 88, 58, 225, 42, 50, 178, 100, 17, 18, 128, 220, 237, 192, 247, 67, 173, 13, 185, 114, 213, 250, 172, 58, 145, 158, 237, 115, 94, 129, 246, 254, 151, 126, 190, 182, 240, 45, 57, 62 }
2014/11/28 15:03:43 | INFO   | jvm 1    | SESSION KEYGEN:
2014/11/28 15:03:43 | INFO   | jvm 1    | PreMaster Secret:
2014/11/28 15:03:43 | INFO   | jvm 1    | (key bytes not available)
2014/11/28 15:03:43 | INFO   | jvm 1    | Thread-31, handling exception: java.security.ProviderException: java.security.NoSuchAlgorithmException: no such algorithm: SunTls12MasterSecret for provider SunPKCS11-NSSfips
2014/11/28 15:03:43 | INFO   | jvm 1    | %% Invalidated:  [Session-136, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256]
2014/11/28 15:03:43 | INFO   | jvm 1    | Thread-31, SEND TLSv1.2 ALERT:  fatal, description = internal_error
2014/11/28 15:03:43 | INFO   | jvm 1    | Thread-31, WRITE: TLSv1.2 Alert, length = 2
2014/11/28 15:03:43 | INFO   | jvm 1    | Thread-31, called closeSocket()
2014/11/28 15:03:43 | INFO   | jvm 1    | Thread-31, IOException in getSession():  javax.net.ssl.SSLException: java.security.ProviderException: java.security.NoSuchAlgorithmException: no such algorithm: SunTls12MasterSecret for provider SunPKCS11-NSSfips
2014/11/28 15:03:43 | INFO   | jvm 1    | Ignoring unavailable cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
2014/11/28 15:03:43 | INFO   | jvm 1    | Ignoring unavailable cipher suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA
2014/11/28 15:03:43 | INFO   | jvm 1    | Ignoring unavailable cipher suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA
2014/11/28 15:03:43 | INFO   | jvm 1    | Ignoring unavailable cipher suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256
2014/11/28 15:03:43 | INFO   | jvm 1    | Ignoring unavailable cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA256
2014/11/28 15:03:43 | INFO   | jvm 1    | Ignoring unavailable cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
2014/11/28 15:03:43 | INFO   | jvm 1    | Ignoring unavailable cipher suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
2014/11/28 15:03:43 | INFO   | jvm 1    | Ignoring unavailable cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
2014/11/28 15:03:43 | INFO   | jvm 1    | Ignoring unavailable cipher suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA
2014/11/28 15:03:43 | INFO   | jvm 1    | Ignoring unavailable cipher suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384
2014/11/28 15:03:43 | INFO   | jvm 1    | Ignoring unavailable cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
2014/11/28 15:03:43 | INFO   | jvm 1    | Ignoring unavailable cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA
2014/11/28 15:03:43 | INFO   | jvm 1    | Ignoring unavailable cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA
2014/11/28 15:03:43 | INFO   | jvm 1    | Ignoring unavailable cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384
2014/11/28 15:03:43 | INFO   | jvm 1    | Fri Nov 28 15:03:43 IST 2014|WARNING|Thread-601|ccs.comp.clientproxy.ProxiedClientListener$ProxyConnection.run
2014/11/28 15:03:43 | INFO   | jvm 1    |       Error processing requests from proxied client : unestablished
2014/11/28 15:03:43 | INFO   | jvm 1    |       Caused by: java.security.NoSuchAlgorithmException: no such algorithm: SunTls12MasterSecret for provider SunPKCS11-NSSfips (java.security.ProviderException); Root cause: no such algorithm: SunTls12MasterSecret for provider SunPKCS11-NSSfips (java.security.NoSuchAlgorithmException)
2014/11/28 15:03:43 | INFO   | jvm 1    |       javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLException: java.security.ProviderException: java.security.NoSuchAlgorithmException: no such algorithm: SunTls12MasterSecret for provider SunPKCS11-NSSfips
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.ssl.SSLSocketImpl.checkEOF(SSLSocketImpl.java:1476)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.ssl.AppInputStream.read(AppInputStream.java:92)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.ssl.AppInputStream.read(AppInputStream.java:69)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at java.io.DataInputStream.readByte(DataInputStream.java:265)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at ccs.comp.clientproxy.ProxiedClientListener$ProxyConnection.getNext(ProxiedClientListener.java:438)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at ccs.comp.clientproxy.ProxiedClientListener$ProxyConnection.run(ProxiedClientListener.java:245)
2014/11/28 15:03:43 | INFO   | jvm 1    |       Caused by: javax.net.ssl.SSLException: java.security.ProviderException: java.security.NoSuchAlgorithmException: no such algorithm: SunTls12MasterSecret for provider SunPKCS11-NSSfips
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1884)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1842)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1825)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1346)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.ssl.SSLSocketImpl.getSession(SSLSocketImpl.java:2171)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at ccs.comp.clientproxy.ProxiedClientListener.listen(ProxiedClientListener.java:126)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at ccs.comp.clientproxy.ProxiedClientListener.run(ProxiedClientListener.java:105)
2014/11/28 15:03:43 | INFO   | jvm 1    |       Caused by: java.security.ProviderException: java.security.NoSuchAlgorithmException: no such algorithm: SunTls12MasterSecret for provider SunPKCS11-NSSfips
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.ssl.Handshaker.calculateMasterSecret(Handshaker.java:1060)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.ssl.Handshaker.calculateKeys(Handshaker.java:999)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:234)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.ssl.Handshaker.process_record(Handshaker.java:804)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1016)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
2014/11/28 15:03:43 | INFO   | jvm 1    |               ... 3 more
2014/11/28 15:03:43 | INFO   | jvm 1    |       Caused by: java.security.NoSuchAlgorithmException: no such algorithm: SunTls12MasterSecret for provider SunPKCS11-NSSfips
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.jca.GetInstance.getService(GetInstance.java:100)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at javax.crypto.JceSecurity.getInstance(JceSecurity.java:109)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at javax.crypto.KeyGenerator.getInstance(KeyGenerator.java:287)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.ssl.JsseJce.getKeyGenerator(JsseJce.java:269)
2014/11/28 15:03:43 | INFO   | jvm 1    |               at sun.security.ssl.Handshaker.calculateMasterSecret(Handshaker.java:1052)
2014/11/28 15:03:43 | INFO   | jvm 1    |               ... 10 more
Strudel answered 7/12, 2014 at 18:14 Comment(0)
E
1

Since the server does not support TLS 1.2, I disabled TLS 1.2 on the client side.

I could not figure out why the handshake fails "even after" disabling TLS 1.2 on the client side. Can you please help me with this?

And:

...
2014/11/28 15:03:43 | INFO   | jvm 1    | Thread-31, WRITE: TLSv1.2 Alert, length = 2
2014/11/28 15:03:43 | INFO   | jvm 1    | Thread-31, called closeSocket()

Just guessing, the the PRF (pseudorandom function) in TLS 1.1 and earlier use MD5 (and SHA1). The TLS 1.2 PRF uses SHA2 family (IIRC). The library is probably being over-zealous with respect to MD5.

Its kind of like trying to get half-pregnant. How do you allow MD5 internally for one function, but not allow it on other functions and still pass a validation with a testing lab?

So you should try to work with the libraries and enable TLS 1.2.

The client is a Java Swing based client application which is launched using Java Web Start.

I think the first step you should perform is drop the Java client that can't cope with TLS 1.2, and verify the server works as expected. That will give you a baseline of sorts.

You can use the following OpenSSL command to test the server with TLS 1.2 enabled:

openssl s_client -tls1_2 -connect www.example.com:443 -servername www.example.com

You can also specify a specific cipher by using the -cipher option. For example, TLS_RSA_WITH_AES_128_CBC_SHA is AES128-SHA in OpenSSL:

openssl s_client -tls1_2 -connect www.example.com:443 -servername www.example.com -cipher AES128-SHA

You can even fecth a page with something like (notice the addition of -ign_eof):

echo -e "GET / HTTP/1.1\r\nHost:www.example.com\r\n\r\n" |  openssl s_client -ign_eof -tls1_2 -connect...

Java is pretty lame when it comes to TLS protocols and cipher suites prior to Java 8. Though available, TLS 1.1 and 1.2 are not enabled by default in Java 7 and earlier. You need to explicitly enable them.

In addition, Java sneaks in SSLv3 even when you don't ask for it. To see for yourself, try SSLContext.getInstance("TLS") and see if SSLv3 is an enabled protocol :)

You can see an example of enabling available protocols and cipher suites at Which Cipher Suites to enable for SSL Socket?

Interestingly, use of MD5 as a PRF in TLS is allowed by NIST. But its a very specific exception. Its allowed because the PRF does not require the property of collision resistance - it just needs to extract entropy.

The following is from NIST's SP 800-135:

The outputs from both P_MD5 and P_SHA-1 are XOR ed together to produce the PRF output. This PRF is used as both a randomness extraction step to generate the master secret and as a key expansion step to derive keying material for the protocol from the master secret.

The TLS 1.0 and 1.1 KDF is approved when the following conditions are satisfied:
(1) The TLS 1.0 and 1.1 KDF is performed in the context of the TLS protocol.
(2) SHA-1 and HMAC are as specified in FIPSs 180-3 and 198-1, respectively.

Estheresthesia answered 7/12, 2014 at 23:13 Comment(1)
(1) Java7 disables 1.1 and 1.2 in client by default, but at least they are available. Before 7 they aren't even implemented. (2) MD5 is FIPS-approved only in limited cases like TLS1.0/1.1 KDF. But the OP's log clearly shows it is trying the TLS1.2 KDF, which is SHA-2, and FAILS because NSS (NOT SunJCE) doesn't support 1.2 -- as he said at the top. – Escamilla

© 2022 - 2024 — McMap. All rights reserved.