How To
ForgeRock Identity Platform
Does not apply to Identity Cloud

How do I collect data for troubleshooting high CPU utilization on AM (All versions) servers?

Last updated Apr 13, 2021

The purpose of this article is to provide troubleshooting guidance if you are experiencing consistently high CPU utilization on the AM server. If CPU utilization is in the range 70-80% under load, you should definitely investigate what is causing it as performance is likely to be impacted.


2 readers recommend this article

Troubleshooting high CPU usage

High CPU utilization is usually experienced in conjunction with other symptoms such as servers becoming unresponsive and poor performance.

Note

On Linux®, Unix® and Solaris® systems, there is a script you can run to capture the stack traces and process information as detailed in How do I use the msnapshots script to capture information for troubleshooting AM (All versions)?

There are a number of different types of data you should collect to troubleshoot high CPU utilization. If you are seeing high CPU usage, you should collect the following data and submit it to us when you raise the ticket to enable us to help you more quickly:

In addition, please include any information about other symptoms you are seeing at the same time as high CPU usage and any changes you have made to your system that may have contributed to the problem. 

Note

It is important that you collect all the data at the point you are experiencing high CPU usage so that the timestamps of the data collected and the high CPU usage correspond. Please notify us of the exact timestamp when high CPU was noticed so we can focus our attention on the relevant information. 

Debug and audit logs

Message level debug logs provide more context for diagnosing errors but can generate large files. You should enable message level debugging when you are experiencing an issue as described in Maintenance Guide › Debug Logging (AM 7 and later) or How do I enable Message level debugging in AM 5.x and 6.x debug files?

Audit log files (Access and Error) record all the AM action attempts, including successful results, and any service errors. They are located in the /path/to/openam/var/audit directory (AM 7 and later) or the /path/to/openAM/log directory (pre-AM 7) by default.

JVM stack traces and current options

JVM stack traces must be collected before killing the affected process or restarting the server; otherwise the information relating to the affected process is lost forever, which may make it impossible to identify the root cause. In addition to collecting stack traces, you should also provide details of the current JVM settings as they can provide an opportunity for tuning, which can reduce CPU usage.

Collecting stack traces and finding current JVM settings is described in How do I collect JVM data for troubleshooting AM (All versions)?

Process / thread information

Detailed process / thread information is useful for identifying the exact thread within a process that is experiencing the high CPU. You should collect this data at the same time as the stack trace so we can look for the identified thread in the stack trace for more information.

  • On Unix and Linux systems, you can use the top command to output process information for all threads to a file: $ top -b -n 1 -H -p [pid] > top-output.txtreplacing [pid] with the process ID of the affected process.
  • On Solaris, you can use the prstat command to capture this information: $ prstat -L -p [pid]replacing [pid] with the process ID of the affected process.
  • On Microsoft® Windows®, you will need to use the Process Monitor.

See How do I find which thread is consuming CPU in a Java process in AM (All versions)? for further information on identifying the thread that is causing high CPU usage.

GC logs

GC logs also provide useful information about tuning issues that may be contributing to the high CPU usage. Enabling GC logging is described in How do I enable Garbage Collector (GC) Logging for AM (All versions)?

Once you have collected your GC logs, you should refer to Best practice for JVM Tuning with G1 GC or Best practice for JVM Tuning with CMS GC for advice on resolving common issues identified in the logs.

See Also

How do I diagnose a hung AM (All versions) server?

Where can I find useful logs for troubleshooting ForgeRock products?

FAQ: AM performance and tuning

Troubleshooting AM and Agents

Performance tuning and monitoring ForgeRock products

Related Training

N/A

Related Issue Tracker IDs

N/A


Copyright and Trademarks Copyright © 2021 ForgeRock, all rights reserved.