How To
ForgeRock Identity Platform
Does not apply to Identity Cloud

How do I collect data for troubleshooting high CPU utilization on IDM (All versions) servers?

Last updated Apr 8, 2021

The purpose of this article is to provide troubleshooting guidance if you are experiencing consistently high CPU utilization on the IDM server. If CPU utilization is in the range 70-80% under load, you should definitely investigate what is causing it as performance is likely to be impacted.


1 reader recommends this article

Troubleshooting high CPU usage

High CPU utilization is usually experienced in conjunction with other symptoms such as servers becoming unresponsive and poor performance.

Note

On Linux®, Unix® and Solaris® systems, there is a script you can run to capture the stack traces and process information as detailed in How do I use the msnapshots script to capture information for troubleshooting IDM (All versions)?

There are a number of different types of data you should collect to troubleshoot high CPU utilization. If you are seeing high CPU usage, you should collect the following data and submit it to us when you raise the ticket to enable us to help you more quickly:

In addition, please include any information about other symptoms you are seeing at the same time as high CPU usage and any changes you have made to your system that may have contributed to the problem. 

Note

It is important that you collect all the data at the point you are experiencing high CPU usage so that the timestamps of the data collected and the high CPU usage correspond. Please notify us of the exact timestamp when high CPU was noticed so we can focus our attention on the relevant information.

Server and audit logs

FINEST level server logs record all system and custom log messages for the server and are useful for troubleshooting. You can enable FINEST level logging as described in Monitoring Guide › Configure Server Logs. In IDM 6.5 and later, threadIDs are included in log messages by default for enhanced troubleshooting. You can add them in earlier versions as described in How do I add Thread IDs to log statements in IDM 5.x and 6?

Audit logs record all system activity into three separate files (access, activity and reconciliation), which can provide useful information on the activities that may be contributing to the high CPU. See Audit Guide › Configure Audit Logging  for further information.

JVM stack traces and current options

JVM stack traces must be collected before killing the affected process or restarting the server; otherwise the information relating to the affected process is lost forever, which may make it impossible to identify the root cause. In addition to collecting stack traces, you should also provide details of the current JVM settings as they can provide an opportunity for tuning, which can reduce CPU usage.

Collecting stack traces and finding current JVM settings is described in How do I collect JVM data for troubleshooting IDM (All versions)?

Process / thread information

Detailed process / thread information is useful for identifying the exact thread within a process that is experiencing the high CPU. You should collect this data at the same time as the stack trace so we can look for the identified thread in the stack trace for more information.

  • On Unix and Linux systems, you can use the top command to output process information for all threads to a file: $ top -b -n 1 -H -p [pid] > top-output.txtreplacing [pid] with the process ID of the affected process.
  • On Solaris, you can use the prstat command to capture this information: $ prstat -L -p [pid]replacing [pid] with the process ID of the affected process.
  • On Microsoft® Windows®, you will need to use the Process Monitor.

See How do I find which thread is consuming CPU in a Java process in IDM (All versions)? for further information on identifying the thread that is causing high CPU usage.

GC logs

GC logs also provide useful information about tuning issues that may be contributing to the high CPU usage. Enabling GC logging is described in How do I enable Garbage Collector (GC) Logging for IDM (All versions)?

Once you have collected your GC logs, you should refer to Best practice for JVM Tuning with G1 GC or Best practice for JVM Tuning with CMS GC for advice on resolving common issues identified in the logs.

See Also

Where can I find useful logs for troubleshooting ForgeRock products?

FAQ: IDM performance and tuning

Troubleshooting IDM

Performance tuning and monitoring ForgeRock products

Related Training

N/A

Related Issue Tracker IDs

N/A


Copyright and Trademarks Copyright © 2021 ForgeRock, all rights reserved.