Solutions
ForgeRock Identity Platform
Does not apply to Identity Cloud

WDSSO authentication fails with GSSException: Failure unspecified at GSS-API level error in AM 5.x and 6.x

Last updated Apr 13, 2021

The purpose of this article is to provide assistance if Windows Desktop SSO (WDSSO) authentication fails with a "GSSException: Failure unspecified at GSS-API level (Mechanism level: Encryption type AES256 CTS mode with HMAC SHA1-96 is not supported/enabled)" error in AM.


1 reader recommends this article

Symptoms

The following error is shown in the Authentication debug log when authentication fails:

amAuthWindowsDesktopSSO:12/10/2016 10:02:47:186 AM GMT: Thread[http-nio-8082-exec-8,5,main]: TransactionId[5abc2e7a-5281-477d-8f2e-afd6b4a51cf9-132] ERROR: Authentication failed with PrivilegedActionException wrapped GSSException. Stack Trace GSSException: Failure unspecified at GSS-API level (Mechanism level: Encryption type AES256 CTS mode with HMAC SHA1-96 is not supported/enabled)   at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Unknown Source)    at sun.security.jgss.GSSContextImpl.acceptSecContext(Unknown Source)    at sun.security.jgss.GSSContextImpl.acceptSecContext(Unknown Source)    at com.sun.identity.authentication.modules.windowsdesktopsso.WindowsDesktopSSO$1.run(WindowsDesktopSSO.java:265) ... Caused by: KrbException: Encryption type AES256 CTS mode with HMAC SHA1-96 is not supported/enabled    at sun.security.krb5.EncryptionKey.findKey(Unknown Source)    at sun.security.krb5.KrbApReq.authenticate(Unknown Source)    at sun.security.krb5.KrbApReq.<init>(Unknown Source)    at sun.security.jgss.krb5.InitSecContextToken.<init>(Unknown Source)    ... 58 more

Recent Changes

The keytab file was created using a key with 256-bit AES encryption, for example with the following crypto option:

-crypto AES256-SHA1

Causes

256-bit AES encryption was not enabled on the machine where the keytab file was created. Java® 8 and earlier does not support 256-bit AES encryption by default; only 128-bit AES encryption is supported.

Solution

This issue can be resolved by installing the Oracle® Java JCE unlimited strength jars in the $JAVA_HOME/jre/lib/security/ directory and your Microsoft® Windows® machine must also support this encryption. These jars can be downloaded from the following link for Java 8 and earlier: Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files Download. Java 9 and later uses the unlimited policy files by default.

You should then re-create the keytab file.

Note

You can check that the keytab file has AES256 enabled as detailed in How do I troubleshoot Kerberos and WDSSO issues in AM (All versions)? (Checking keytab file details).

See Also

WDSSO/Kerberos authentication fails in AM (All versions) with an HTTP 400 Bad Request response

Unable to obtain password from user error when Kerberos authentication fails in AM (All versions)

Clock skew too great (37) error when WDSSO authentication fails in AM (All versions)

How do I troubleshoot Kerberos and WDSSO issues in AM (All versions)?

Configuring and troubleshooting WDSSO in AM

Authentication and Single Sign-On Guide › Windows Desktop SSO Authentication Module

Related Training

N/A

Related Issue Tracker IDs

N/A


Copyright and Trademarks Copyright © 2021 ForgeRock, all rights reserved.