How do I collect data for troubleshooting high CPU utilization or Out of Memory errors on IG (All versions) servers?

Troubleshooting high CPU usage or Out of Memory errors

Both high CPU utilization and Out of Memory errors are usually experienced in conjunction with other symptoms such as servers becoming unresponsive and poor performance.

There are a number of different types of data you should collect to troubleshoot high CPU utilization or Out of Memory errors; you should collect the following data and submit it to us when you raise the ticket to enable us to help you more quickly:

• Debug logs
• Configuration directory
• JVM stack traces and current options
• Process / thread information
• Garbage Collector (GC) logs

In addition, please include any information about other symptoms you are seeing at the same time as the issue and any changes you have made to your system that may have contributed to the problem. 

Debug logs

Debug level debug logs provide more context for diagnosing errors but can generate large files. You should collect logs when you are experiencing an issue as detailed in How do I generate more detailed debug logs to diagnose an issue in IG (All versions)?

Configuration directory

The configuration directory ($HOME/.openig/config/) contains all the configuration settings for an IG server instance and is useful for checking settings that may play a role in the issue.

JVM stack traces and current options

JVM stack traces must be collected before killing the affected process or restarting the server; otherwise the information relating to the affected process is lost forever, which may make it impossible to identify the root cause. In addition to collecting stack traces, you should also provide details of the current JVM settings as they can provide an opportunity for tuning, which can reduce CPU usage or Out of Memory errors.

Collecting stack traces and finding current JVM settings is described in How do I collect JVM data for troubleshooting IG (All versions)? This information only applies if IG is running in a web container.

Process / thread information

Detailed process / thread information is useful for identifying the exact thread within a process that is experiencing high CPU. You should collect this data at the same time as the stack trace so we can look for the identified thread in the stack trace for more information. This information only applies if IG is running in a web container.

• On Unix and Linux systems, you can use the top command to output process information for all threads to a file: $ top -b -n 1 -H -p [pid] > top-output.txtreplacing [pid] with the process ID of the affected process.
• On Solaris, you can use the prstat command to capture this information: $ prstat -L -p [pid]replacing [pid] with the process ID of the affected process.
• On Microsoft® Windows®, you will need to use the Process Monitor.

See How do I find which thread is consuming CPU in a Java process in IG (All versions)? for further information on identifying the thread that is causing high CPU usage.

GC logs

GC logs also provide useful information about tuning issues that may be contributing to the high CPU usage or Out of Memory errors. Enabling GC logging is described in How do I enable Garbage Collector (GC) Logging for IG? This information only applies if IG is running in a web container.

Once you have collected your GC logs, you should refer to Best practice for JVM Tuning with G1 GC or Best practice for JVM Tuning with CMS GC for advice on resolving common issues identified in the logs.

See Also

Where can I find useful logs for troubleshooting ForgeRock products?

Troubleshooting IG

Performance tuning and monitoring ForgeRock products

Troubleshooting

Related Training

N/A

Related Issue Tracker IDs

N/A