Attempting to authenticate a windows client (IE/Firefox) via SPNEGO and Kerberos. The server side is Java/Tomcat with JCIFS for SPNEGO authentication. The SSO (Kerberos) auth works fine when hosting the server side on a Win 2008 R2 server. However, when on a 2012 server it fails with a GSSException: Defective token detected.
Digging a bit deeper with network tracing I found that, in the working case the IE client sends the negotiate tokens with 4 mechTypes :
- 1.2.840.48018.1.2.2 -
MS KRB5, - 1.2.840.113554.1.2.2 -
KRB5, - 1.3.6.1.4.1.311.2.2.30 -
NEGOEX, and - 1.3.6.1.4.1.311.2.2.10 -
NTLMSSP
In this case my server side will complete the SPNEGO selecting MS KRB5. However, in the problem case, the IE client only sends token with 2 mechtypes - NEGOEX and NTLMSSP. And this is initiator preferred. Java doesn't support NEGOEX and hence it fails.
Some search revealed that this problem is associated with bugs in JDK*, or otherwise issues with DNS. However, I'm on latest JDK and DNS seems to be okay. So my question is, when does a browser in Windows switch to NEGOEX in SPNEGO and why ? The closest answer I found was in an msdn blog which says Kerberos is not available since it's not in a domain environment. However, the client is indeed in domain environment and klist shows a valid Kerberos ticket. If it indeed is a domain problem, what exactly could be the root cause, and how can I avoid the problem ?
Footnote, some background research information : JDK8 has seen many fixes in the GSS mechanism. There were things broken in jdk8u40 and jdk8u45. Then further fixes are present in jdku65. A bug report which was supposed to implement NEGOEX was closed with a fix
"a fix to SPNEGO that allows NEGOEX be presented and bypassed"
however, I'm not sure if NEGOEX is indeed working. The NEGOEX IETF standard also looks abandoned with the draft RFC in the expired state. So I doubt if it will really be supported by Java, libraries.