How do I find which thread is consuming CPU in a Java process in IG 6.x and 7.x?
The purpose of this article is to provide assistance on identifying which thread is consuming CPU in a Java® process in IG when it is running in a web container. This is a useful troubleshooting technique when you are experiencing high CPU under load.
Note
This article does not apply to IG 2023.2 and later because you cannot run IG in a web container in these later versions.
Identifying the thread
You can identify which thread is consuming CPU in a Java process as follows:
- Determine the process ID (PID) of the web container instance in which the affected server is running using the following command: $ jps -v
- Determine which thread in the PID identified in step 1 is consuming the CPU:
- On Unix® and Linux® systems, you can use the top command: $ top -n 1 -H -p [pid]replacing [pid] with the process ID of the affected process.
- On Solaris®, you can use the prstat command: $ prstat -L -p [pid]replacing [pid] with the process ID of the affected process.
- On Microsoft® Windows®, you can use the Process Monitor.
These commands all give an output listing all the threads in the selected process along with various details, including the CPU usage percentage. For example, the top command gives an output similar to this:
Tasks: 152 total, 0 running, 152 sleeping, 0 stopped, 0 zombie Cpu(s): 2.0%us, 1.4%sy, 0.0%ni, 94.7%id, 0.8%wa, 0.0%hi, 1.1%si, 0.0%st Mem: 5008484k total, 4854428k used, 154056k free, 85624k buffers Swap: 5013496k total, 6516k used, 5006980k free, 3393948k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 8706 user.1 20 0 2433m 561m 10m R 32.0 11.5 0:35.45 java 8624 user.1 20 0 2433m 561m 10m S 0.0 11.5 0:00.01 java 8625 user.1 20 0 2433m 561m 10m S 0.0 11.5 0:10.12 java 8626 user.1 20 0 2433m 561m 10m S 0.0 11.5 0:35.60 java 8627 user.1 20 0 2433m 561m 10m S 0.0 11.5 0:00.01 java 8628 user.1 20 0 2433m 561m 10m S 0.0 11.5 0:00.02 java
In this example, we can see that thread ID 8706 is consuming 32% of the CPU.
- Produce a stack trace for the PID identified in step 1 using one of the following commands:
- jstack command: $ jstack [PID] > jstack-output.txt
- kill -3 command (Unix® and Linux® systems only), which consistently produces better results: $ kill -3 [pid]The kill -3 command outputs the stack trace to the log file where the standard output is directed (typically the error log file). The kill -3 command does not stop the running process; it will continue to run after outputting the stack trace.
- Convert the thread ID identified as problematic in step 2 to a hex value. In our example output, the thread ID decimal value 8706 converts to hex value 2202.
- Search the stack trace output for this hex value using grep. You are looking for a thread nid that matches this hex value: $ cat jstack-output.txt | grep -i 2202This command gives an output similar to this: "Concurrent Mark-Sweep GC Thread" prio=10 tid=0x00007f1448029800 nid=0x2202 runnableIn this example, we can see that the problem is a Garbage Collector (GC) thread, which suggests that GC is not properly tuned.
See Also
How do I collect JVM data for troubleshooting IG 6.x and 7.x?
How do I enable Garbage Collector (GC) Logging for IG?
Related Training
N/A
Related Issue Tracker IDs
N/A