How To
ForgeRock Identity Platform
Does not apply to Identity Cloud

How do I collect data for troubleshooting high CPU utilization on DS (All versions) servers?

Last updated Jan 12, 2023

The purpose of this article is to provide troubleshooting guidance if you are experiencing consistently high CPU utilization on the DS server. If CPU utilization is sustained in the 70-80% range or higher under load, you should investigate to see what is causing this increase as performance is likely to be impacted.


1 reader recommends this article

Troubleshooting high CPU usage

Sustained high CPU utilization is usually experienced in conjunction with other symptoms such as slower response times under load, servers becoming unresponsive and appearing to be out of memory or in hang state, a decrease in overall server performance, etc.

Note

You can run the Support Extract tool to make it easier to capture the data listed below as detailed in How do I use the Support Extract tool in DS 6.5.x and 7.x to capture troubleshooting data?

There are a number of different types of data you should collect to troubleshoot high CPU utilization. If you are seeing high CPU, you should collect the following data and submit it when raising the ticket to help us troubleshoot your issue more quickly.

In addition, please include any information about other symptoms you are seeing at the same time as high CPU usage and any changes you have made to your system that may have contributed to the problem.

Note

It is necessary to collect the data at the point when the server is exhibiting high CPU so the timestamps across the data set correspond with the time high CPU occurred. Please notify us of the exact timestamp when high CPU was noticed so we can focus our attention on the relevant information.

Access log file

The Access log file records details about all the operations processed by the server. This file is located in the /path/to/ds/logs directory where DS is installed.

See How do I use the Access log to troubleshoot DS (All versions)? for further information about this log file and advice on resolving common issues identified in the logs.

Configuration file

The configuration file (config.ldif, which is located in the /path/to/ds/config directory where DS is installed) contains all the configuration settings for a DS server instance and is useful for checking your configuration, including index settings.

See How do I troubleshoot issues with my indexes in DS (All versions)? for further information on troubleshooting indexes. One common index issue that contributes to high CPU is unindexed searches; see Unindexed searches causing slow searches and poor performance on DS (All versions) server for further information.

JVM stack traces and current options

JVM stack traces must be collected before killing the affected process or restarting the server; otherwise the information relating to the affected process is lost forever, which may make it impossible to identify the root cause. In addition to collecting stack traces, you should also provide details of the current JVM settings as they can provide an opportunity for tuning, which can reduce CPU usage.

Collecting stack traces and finding current JVM settings is described in How do I collect JVM data for troubleshooting DS?

Process / thread information

Detailed process / thread information is useful for identifying the exact thread within a process that is experiencing high CPU. You should collect this data at the same time as the stack trace so we can look for the identified thread in the stack trace for more information.

  • On Unix® and Linux® systems, you can use the top command to output process information for all threads to a file: $ top -b -n 1 -H -p [pid] > top-output.txtreplacing [pid] with the process ID of the affected process.
  • On Solaris®, you can use the prstat command to capture this information: $ prstat -L -p [pid]replacing [pid] with the process ID of the affected process.
  • On Microsoft® Windows®, you will need to use the Process Monitor.

See How do I find which thread is consuming CPU in a Java process in DS? for further information on identifying the thread that is causing high CPU usage.

GC logs

GC logs also provide useful information about tuning issues that may be contributing to the high CPU usage. Enabling GC logging is described in How do I enable Garbage Collector (GC) Logging for DS?

Once you have collected your GC logs, you should refer to Best practice for JVM Tuning with G1 GC or Best practice for JVM Tuning with CMS GC for advice on resolving common issues identified in the logs.

See Also

Where can I find useful logs for troubleshooting ForgeRock products?

FAQ: DS performance and tuning

Troubleshooting DS

Performance tuning and monitoring ForgeRock products

About Logs

Related Training

N/A

Related Issue Tracker IDs

N/A


Copyright and Trademarks Copyright © 2023 ForgeRock, all rights reserved.