Unhandled exception: Internal Server Error (500) when running OIDC_CLAIMS scripts under load in AM (All versions)

Last updated Jan 16, 2023

The purpose of this article is to provide assistance if you encounter "Unhandled exception: Internal Server Error (500) - The server encountered an unexpected condition which prevented it from fulfilling the request" when running OIDC_CLAIMS scripts under load in AM. Examples of this may be performance testing concurrent logins with repeated requests to the OIDC userinfo endpoint or load testing with repeated requests to the access_token endpoint.

Symptoms

You receive the following 400 response to the request:

{"code":400,"reason":"The request could not be understood by the server due to malformed syntax"}

The following error is shown in the OAuth2Provider debug log when this happens:

OAuth2Provider:10/01/2019 09:43:39:492 AM GMT: Thread[http-nio-8080-exec-40,5,main]: TransactionId[50321dff-a328-4d40-9527-9fc53d1740b1-21332] ERROR: Unhandled exception: Internal Server Error (500) - The server encountered an unexpected condition which prevented it from fulfilling the request at org.restlet.resource.ServerResource.doHandle(ServerResource.java:539) at org.restlet.resource.ServerResource.get(ServerResource.java:742) at org.restlet.resource.ServerResource.doHandle(ServerResource.java:617) at org.restlet.resource.ServerResource.doNegotiatedHandle(ServerResource.java:678) at org.restlet.resource.ServerResource.doConditionalHandle(ServerResource.java:356) at org.restlet.resource.ServerResource.handle(ServerResource.java:1043) ... Caused by: java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3aba1094 rejected from java.util.concurrent.ThreadPoolExecutor@4552b7f2[Running, pool size = 50, active threads = 50, queued tasks = 10, completed tasks = 204] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)

The error logging has been improved in AM 7 and you will see the following message:

Execution of script rejected from thread pool - consider adjusting the configuration for scriptName script thread pool

Recent Changes

N/A

Causes

AM does not have a sufficient number of threads available to run the scripts, which causes the scripts to be blocked. The threads available are determined by two settings: Maximum thread pool size and Thread pool queue size.

Solution

This issue can be resolved by tuning these thread settings appropriately for your environment. It is recommended you set Maximum thread pool size to 300 and tune Thread pool queue size to a value that works for your anticipated volumes. See Engine configuration for further information on these settings.

Tuning these thread settings is outside the scope of ForgeRock support; if you want more tailored advice, consider engaging Deployment Support Services.

You can tune these settings using either the AM admin UI, Amster or ssoadm:

AM admin UI: navigate to: Configure > Global Services > Scripting > Secondary Configurations > [Script Type] > Secondary Configurations > EngineConfiguration and enter new values for Maximum thread pool size and Thread pool queue size.
Amster: follow the steps in How do I update property values in AM (All versions) using Amster? with these values:
- Entity: ScriptingEngineConfiguration
- Property: maxThreads and queueSize
ssoadm: enter the following command: $ ./ssoadm set-sub-cfg -s ScriptingService -g OIDC_CLAIMS/engineConfiguration -u [adminID] -f [passwordfile] -o set -a maxThreads=[maxThreads] queueSize=[queueSize] replacing [adminID], [passwordfile], [maxThreads] and [queueSize] with appropriate values.

Note

You must restart the web application container in which AM runs to apply these configuration changes.

Related Training

N/A

Related Issue Tracker IDs

OPENAM-11778 (Getting accessToken using authorization_code result in Unhandled exception)