Solutions

Unhandled exception: Internal Server Error (500) when running OIDC_CLAIMS scripts under load in AM (All versions)

Last updated Sep 15, 2020

The purpose of this article is to provide assistance if you encounter "Unhandled exception: Internal Server Error (500) - The server encountered an unexpected condition which prevented it from fulfilling the request" when running OIDC_CLAIMS scripts under load in AM. Examples of this may be performance testing concurrent logins with repeated requests to the OIDC userinfo endpoint or load testing with repeated requests to the access_token endpoint.


Symptoms

You receive the following 400 response to the request:

{"code":400,"reason":"The request could not be understood by the server due to malformed syntax"}

The following error is shown in the OAuth2Provider debug log when this happens:

OAuth2Provider:10/01/2019 09:43:39:492 AM GMT: Thread[http-nio-8080-exec-40,5,main]: TransactionId[50321dff-a328-4d40-9527-9fc53d1740b1-21332]
ERROR: Unhandled exception: 
Internal Server Error (500) - The server encountered an unexpected condition which prevented it from fulfilling the request
   at org.restlet.resource.ServerResource.doHandle(ServerResource.java:539)
   at org.restlet.resource.ServerResource.get(ServerResource.java:742)
   at org.restlet.resource.ServerResource.doHandle(ServerResource.java:617)
   at org.restlet.resource.ServerResource.doNegotiatedHandle(ServerResource.java:678)
   at org.restlet.resource.ServerResource.doConditionalHandle(ServerResource.java:356)
   at org.restlet.resource.ServerResource.handle(ServerResource.java:1043)
...
Caused by: java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3aba1094 rejected from java.util.concurrent.ThreadPoolExecutor@4552b7f2[Running, pool size = 50, active threads = 50, queued tasks = 10, completed tasks = 204]
   at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)

The error logging has been improved in AM 7 and you will see the following message:

Execution of script rejected from thread pool - consider adjusting the configuration for scriptName script thread pool

Recent Changes

N/A

Causes

AM does not have a sufficient number of threads available to run the scripts, which causes the scripts to be blocked. The threads available are determined by two settings: Maximum thread pool size and Thread pool queue size as described here: Reference › Engine Configuration.

Note

There is a known issue with the documentation where it incorrectly states the recommended value for Thread pool queue size is -1: OPENAM-16478 (Thread pool queue size should recommend tuning this setting rather than setting it to -1).

Solution

This issue can be resolved by tuning these thread settings appropriately for your environment. It is recommended you set Maximum thread pool size to 300 and tune Thread pool queue size to a value that works for your anticipated volumes. Tuning these thread settings is outside the scope of ForgeRock support; if you want more tailored advice, consider engaging Deployment Support Services.

You can tune these settings using either the console, Amster or ssoadm:

  • AM console: navigate to: Configure > Global Services > Scripting > Secondary Configurations > [Script Type] > Secondary Configurations > EngineConfiguration and enter new values for Maximum thread pool size and Thread pool queue size.
  • Amster: follow the steps in How do I update property values in AM (All versions) using Amster? with these values:
    • Entity: ScriptingEngineConfiguration
    • Property: maxThreads and queueSize
  • ssoadm: enter the following command:
    $ ./ssoadm set-sub-cfg -s ScriptingService -g OIDC_CLAIMS/engineConfiguration -u [adminID] -f [passwordfile] -o set -a maxThreads=[maxThreads] queueSize=[queueSize] 
    replacing [adminID], [passwordfile], [maxThreads] and [queueSize] with appropriate values.
Note

You must restart the web application container in which AM runs to apply these configuration changes.

See Also

How do I add custom claims to the OIDC Claims Script in AM/OpenAM (All versions)?

OAuth 2.0 in AM/OpenAM

Performance tuning and monitoring ForgeRock products

OpenID Connect 1.0 Guide

Related Training

N/A

Related Issue Tracker IDs

OPENAM-13270 (EngineConfiguration settings need clarification)

OPENAM-11778 (Getting accessToken using authorization_code result in Unhandled exception)

OPENAM-9307 (Significant number of internal errors while generating access_token load)



Copyright and TrademarksCopyright © 2020 ForgeRock, all rights reserved.
Loading...