Authenticating with Windows Desktop SSO in AM (All versions) does not proceed when using a non-Microsoft Edge browser

Last updated Jan 16, 2023

The purpose of this article is to provide assistance when authenticating with the Windows Desktop SSO (WDSSO) authentication module in AM does not proceed when using a browser other than Microsoft Edge, such as Chrome™ or Firefox®. The user sees a "401 Unauthorized / Access denied" error.

1 reader recommends this article

Symptoms

The user sees a 401 Unauthorized / Access denied error when they attempt to access a resource protected by AM, even though they are logged in to Microsoft® Windows® and another authentication module exists in the authentication chain.

An error similar to the following is shown in the Authentication log:

amLoginModule:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] Login, class = com.sun.identity.authentication.modules.windowsdesktopsso.WindowsDesktopSSO, module=WDSSO, file=/config/auth/default/WindowsDesktopSSO.xml amLoginModule:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] AMLoginModuleiplanet-am-auth-shared-state-behavior-pattern is set to tryFirstPass amLoginModule:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] This module is not done yet. CurrentState: 1 amLoginModule:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] callback stateLength in file = 1 amLoginModule:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] callback size for state 1=1 amLoginModule:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] clone #0 is PagePropertiesCallback amLoginModule:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] Login, state = 1 amCallback:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] callback handler method amAuth:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] Setting page timeout :120 amAuth:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] setting Last Callback Sent :1394578342563 amCallback:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] Set callbacks, throwing java.lang.Error. amJAAS:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] LoginContext.invoke():Handling expected java.lang.Error amAuth:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] Caught java.lang.Error returned from DSAMEHandler java.lang.Error: return from DSAMECallback at com.sun.identity.authentication.service.DSAMECallbackHandler.handle(DSAMECallbackHandler.java:141) at com.sun.identity.authentication.spi.AMLoginModule.login(AMLoginModule.java:1134) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) ... amAuth:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] AMLoginContext:Thread started... returning. amAuthContextLocal:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] after AMLoginContext::exceuteLogin : amAuth:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] getStatus : status is... : 2 amAuth:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] getStatus : status is... : 2 amAuthContextLocal:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] Status at the end of login() : in_progress amAuthContextLocal:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] AuthContextLocal::hasMoreRequirements() amAuth:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] getStatus : status is... : 2 amAuth:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] getStatus : status is... : 2 amAuth:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] Recd Callback in amlc.getRequiredInfo : com.sun.identity.authentication.spi.PagePropertiesCallback@2f466880 amAuth:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] Recd Callback in amlc.getRequiredInfo : com.sun.identity.authentication.spi.HttpCallback@3776c3bf amLoginViewBean:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] In getLoginDisplay, has More Requirements amAuthContextLocal:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] AuthContextLocal::getRequirements() amAuth:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] getStatus : status is... : 2 amAuth:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] getStatus : status is... : 2 amLoginViewBean:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] Start authorization negotiation... amLoginViewBean:03/11/2014 11:52:22:563 PM CET: Thread[http-bio-8080-exec-8,5,main] header: WWW-Authenticate, value: Negotiate, code: 401

Recent Changes

Implemented the Windows Desktop SSO authentication module.

Causes

The browser cannot return a Kerberos™ token during the SPNEGO handshake because it cannot retrieve a Kerberos service ticket for AM from Active Directory®.

Two common reasons for the browser failing to send a Kerberos token are:

The AM FQDN is not listed as a trusted host in the browser.
The Service Principal Name (SPN) is not set up correctly in Active Directory.

Solution

This issue can be resolved by checking the following:

Check if the Kerberos service ticket was retrieved for AM using the Klist utility from Microsoft:
- If the Kerberos service ticket does exist for the AM FQDN - check your browser settings to ensure the browser is correctly set up for SPNEGO (Integrated Windows Authentication) and update where necessary. See How do I set up Kerberos authentication in AM (All versions)? for further information.
- If the Kerberos service ticket does not exist for the AM FQDN - proceed to the next step.

For example, running the klist command:$ klist

Produces an output such as the following:Server: HTTPS/am.example.com @ FORGEROCK.COM KerbTicket Encryption Type: RSADSI RC4-HMAC(NT) Ticket Flags 0x40a10000 -> forwardable renewable pre_authent name_canonicalize Start Time: 3/17/2016 11:33:06 (local) End Time: 3/17/2016 21:31:54 (local) Renew Time: 3/24/2016 11:31:54 (local) Session Key Type: RSADSI RC4-HMAC(NT) Cache Flags: 0 Kdc Called: SVR1

Check the AM FQDN is listed as a trusted host in the browser and add it if it's not listed. You can check as follows according to which browser you are using:
- Chrome - since Chrome uses the Microsoft Edge settings, you should check the AM FQDN is added to the trusted site list in Microsoft Edge (Security tab > Trusted Sites > Sites).
- Firefox - the AM FQDN should be added to the network.negotiate-auth.trusted-uris and network.automatic-ntlm-auth.trusted-uris settings (accessed via about:config).
Check the SPN for the AM service in Active Directory and correct if necessary. Ensure the FQDN (which matches the SPN in AD) is used and the specified authcontext matches the auth module/chain for WDSSO. You can use the following command to check the SPN: $ setspn -l [account]replacing [account] with the name of the account created for AM. An example command looks like this: $ setspn -l amand gives an output as follows: Registered ServicePrincipalNames for CN=am,OU=employees,DC=example,DC=com: HTTPS/am.example.com

Note

If none of these resolve the issue, another problem must exist with the Kerberos protocol between the browser and Active Directory; you can attempt to track this down using utilities such as the Microsoft Network Monitor or the Microsoft Message Analyzer.

Related Training

N/A

Related Issue Tracker IDs

N/A