How do I diagnose a hung AM (All versions) server?
The purpose of this article is to provide information on diagnosing a hung AM server, including the data you should collect to send to ForgeRock Support for troubleshooting.
1 reader recommends this article
Diagnosing a hung AM server
There are a number of different types of data that you should collect to help us diagnose a hung AM server. If you suspect you are experiencing a hung AM server, you should collect the following data and submit it to us when you raise the ticket to enable us to help you more quickly:
- Debug logs.
- JVM data.
- Garbage Collector (GC) logs.
- HTTP container logs.
- Native stack traces.
Note
It is important that you collect all the data immediately after reproducing the issue so that the timestamps of the data collected and the suspected hung AM server correspond. Please notify us of the exact timestamp when the issue occurred so we can focus our attention on the relevant information.
Debug logs
Message level debug logs provide more context for diagnosing errors but can generate large files. You should enable message level debugging when you are experiencing an issue as described in Debug logging (AM 7 and later) or How do I enable Message level debugging in AM 6.x debug files?
JVM data
JVM data must be collected before killing the affected process or restarting the server; otherwise the information relating to the affected process is lost forever, which may make it impossible to identify the root cause. There are three types of data that you should collect (stack traces, heap histograms and heap dumps) as well as providing details of the current JVM settings as described in How do I collect JVM data for troubleshooting AM?
GC logs
GC logs are not always crucial at this stage, but can provide useful information about potential tuning issues that may be contributing to your problems. Enabling GC logging is described in How do I enable Garbage Collector (GC) Logging for AM?
HTTP container logs
HTTP container server logs (Access and Error) are very useful for troubleshooting as between them they show all requests to the server, whether they were successful or not and any corresponding error messages. The actual name and location of the server logs are specific to your web container. For example, for Apache Tomcat™, the logs are called localhost_access_log.YYYY-MM-DD.log and catalina.out, and are located in the /path/to/tomcat/logs/ directory by default.
Native stack traces
Native stack traces are not always necessary but can provide useful information when troubleshooting, especially if you can isolate the the affected process. You can use the pstack command for Solaris® and Linux® (although pstack is not installed by default on Linux):
$ pstack -F [pid]replacing [pid] with the process ID of the affected process.
See Also
How do I collect data for troubleshooting high CPU utilization on AM (All versions) servers?
How do I collect all the data required for troubleshooting AM and Agents (All versions)?
Related Training
N/A
Related Issue Tracker IDs
N/A