The following error is shown in the Authentication debug log when authentication fails:amAuthWindowsDesktopSSO:12/10/2016 10:02:47:186 AM GMT: Thread[http-nio-8082-exec-8,5,main]: TransactionId[5abc2e7a-5281-477d-8f2e-afd6b4a51cf9-132] ERROR: Authentication failed with PrivilegedActionException wrapped GSSException. Stack Trace GSSException: Failure unspecified at GSS-API level (Mechanism level: Encryption type AES256 CTS mode with HMAC SHA1-96 is not supported/enabled) at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Unknown Source) at sun.security.jgss.GSSContextImpl.acceptSecContext(Unknown Source) at sun.security.jgss.GSSContextImpl.acceptSecContext(Unknown Source) at com.sun.identity.authentication.modules.windowsdesktopsso.WindowsDesktopSSO$1.run(WindowsDesktopSSO.java:265) ... Caused by: KrbException: Encryption type AES256 CTS mode with HMAC SHA1-96 is not supported/enabled at sun.security.krb5.EncryptionKey.findKey(Unknown Source) at sun.security.krb5.KrbApReq.authenticate(Unknown Source) at sun.security.krb5.KrbApReq.<init>(Unknown Source) at sun.security.jgss.krb5.InitSecContextToken.<init>(Unknown Source) ... 58 more
The keytab file was created using a key with 256-bit AES encryption, for example with the following crypto option:-crypto AES256-SHA1
256-bit AES encryption was not enabled on the machine where the keytab file was created. Java® 8 and earlier does not support 256-bit AES encryption by default; only 128-bit AES encryption is supported.
This issue can be resolved by installing the Oracle® Java JCE unlimited strength jars in the $JAVA_HOME/jre/lib/security/ directory and your Microsoft® Windows® machine must also support this encryption. These jars can be downloaded from the following link for Java 8 and earlier: Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files Download. Java 9 and later uses the unlimited policy files by default.
You should then re-create the keytab file.
You can check that the keytab file has AES256 enabled as detailed in How do I troubleshoot Kerberos and WDSSO issues in AM (All versions)? (Checking keytab file details).