Authentication fails with IDM 5.x or 6.x integrated with AM when session-jwt cookie size exceeds browser limits

Last updated Nov 2, 2020

The purpose of this article is to provide assistance if authentication fails in an environment where IDM is integrated with AM using the OAUTH_CLIENT module (IDM 5.5 and later) or the OPENID_CONNECT module (IDM 5).


Authentication fails.

If you examine network traffic using your browser's Developer Tools or by capturing a HAR file, you will notice the following: 

  • The /openidm/authentication POST call is successful (200 OK response) and you can see the response to this request contains a Set-Cookie for session-jwt. 
  • The session-jwt cookie set is larger than 4096 bytes.
  • The subsequent /openidm/managed/user/userID GET call fails (403 Forbidden response) and the session-jwt cookie is not set (null).

You can capture a HAR file as described in: How do I create a HAR file for troubleshooting IDM/OpenIDM (All versions)?

Recent Changes

Changed the default alias (openidm-localhost) for the encryption key used by the Jetty® web server to service SSL requests.

Assigned additional authorization roles to the user(s) who cannot authenticate.


The session-jwt cookie is larger than 4096 bytes, which is the limit for most browsers per RFC 6265. As a result, IDM sends the session-jwt cookie correctly and it's received by the browser (shown in the POST call), but the browser does not save the cookie nor send it in the next request (shown in the GET call), which causes the authentication flow to fail.

The session-jwt cookie contains a lot of information and certain items can cause it to be too big, including (but not limited to):

  • Public key length, particularly if the public key in use exceeds the default length of 2048 bit.
  • Number of authorization roles, particularly if the user has more than one authorization role assigned to them. 
  • Length of realm names and/or use of sub-realms as this results in longer URLs within the cookie.

Public key length

If you're unsure what the public key length is in your environment, you can check it as follows providing you have Java® keytool and openssl installed: 

  1. Check the alias of the certificate currently in use as follows depending on your version (it's set to openidm-localhost in a default installation):
    • IDM 6.5.x: look at the alias specified for the idm.jwt.session.module.encryption secret in the secrets.json file (located in /path/to/idm/conf): { "secretId" : "idm.jwt.session.module.encryption", "types": [ "ENCRYPT", "DECRYPT" ], "aliases": [ "&{openidm.https.keystore.cert.alias|openidm-localhost}" ] },
    • Pre-IDM 6.5: look for the value of the following property in the file: openidm.https.keystore.cert.alias
  2. Navigate to the /path/to/idm/security directory.
  3. Run the following keytool command to return the public key, ensuring you substitute your alias, password and keystore details as needed: $ keytool -list -rfc -storetype JCEKS -keystore keystore.jceks -storepass changeit -alias openidm-localhost | openssl x509 -inform pem -pubkey This will give you an output similar to this: -----BEGIN PUBLIC KEY----- MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkOgv6oQT5bKk... kwIDAQAB <SNIP> -----END PUBLIC KEY-----
  4. Save this output (including the headers) to a text file.
  5. Run the following command referencing the file you just created: $ openssl rsa -noout -text -inform PEM -in pubkey.txt -pubin This will give you an output similar to this, where the public key length is shown (2048 bit in this example): Public-Key: (2048 bit) Modulus:     00:90:e8:2f:ea:84:13:e5:b2:a4:5a:81:56:05:04:     <SNIP>     5d:93 Exponent: 65537 (0x10001)


This issue can be resolved by reducing the length of the session-jwt cookie to below 4096 bytes.

Depending on what caused the large session-jwt cookie, the following suggestions may help:

See Also

How does the OIDC authorization flow work when IDM 5.5.x or 6.x is integrated with AM?

How do I renew my existing CA certificate in use by IDM/OpenIDM (All versions)?

Integrator's Guide › Generating a Self-Signed Certificate

Related Training


Related Issue Tracker IDs

OPENIDM-11055 (Full Stack: in some configurations, may need to increase the default header size)

OPENAM-13165 (Full Stack: in some configurations, may need to increase the default header size)

Copyright and TrademarksCopyright © 2020 ForgeRock, all rights reserved.