Inspecting Java threads in Linux using top

Asked 21/1, 2011 at 21:50 Answered 29/1, 2019 at 17:37

I am inspecting a Java process in Linux using

top -H

However, I cannot read the name of the thread in the "COMMAND" column (because it is too long). If I use 'c' to expand the full name of the process, then it is still to long to fit.

How can I obtain the full name of the command?

Sunflower answered 21/1, 2011 at 21:50 Comment(5)

will top give java threads info ?? – Eddy 21/1, 2011 at 21:54

That's not the thread but the process. – Faye 21/1, 2011 at 22:0

@OscarRyz, -H tells top to list (native) threads. – Sharpsighted 21/1, 2011 at 22:3

@nhnb That being the case I'll withdraw my answer – Faye 21/1, 2011 at 22:13

On my machine top -H gives all the pids of the threads within a Java process, and the "COMMAND" column gives the Thread's name. – Sunflower 21/1, 2011 at 22:16

You can inspect java threads with the tool jstack. It will list the names, stacktraces and other useful information of all threads belonging to the specified process pid.

Edit: The parameter nid in the thread dump of jstack is the hex version of the LWP that is displayed by top in the pid column for threads.

Sharpsighted answered 21/1, 2011 at 21:55 Comment(2)

I think he meant exactly what he wrote. He even used -H as parameter for "top" to tell "top" to display threads. – Sharpsighted 21/1, 2011 at 22:7

You may need to sudo execute the command with the user executing the java process. – Sulfite 27/5, 2020 at 10:4

This might be a little old, but here's what I did to kinda merge top and jstack together. I used two scripts, but I'm sure it all could be done in one.

First, I save the output of top with the pids for my java threads into a file and save the jstack output into another file:

#!/bin/sh
top -H -b -n 1 | grep java > /tmp/top.log
jstack -l `ps fax | grep java | grep tomcat | sed "s/ *\([0-9]*\) .*/\1/g"` > /tmp/jstack.log

Then I use a perl script to call the bash script (called cpu-java.sh here) and kinda merge the two files (/tmp/top.log and /tmp/jstack.log):

#!/usr/bin/perl
system("sh cpu-java.sh");
open LOG, "/tmp/top.log" or die $!;
print "PID\tCPU\tMem\tJStack Info\n";
while ($l = <LOG>) {
    $pid = $l;
    $pid =~ s/root.*//g;
    $pid =~ s/ *//g;
    $hex_pid = sprintf("%#x", $pid);
    @values = split(/\s{2,}/, $l);
    $pct = $values[4];
    $mem = $values[5];
    open JSTACK, "/tmp/jstack.log" or die $!;   
    while ($j = <JSTACK>){
        if ($j =~ /.*nid=.*/){
            if ($j =~ /.*$hex_pid.*/){
                $j =~ s/\n//;
                $pid =~ s/\n//;
                print $pid . "\t" . $pct . "\t" . $mem . "\t" .  $j . "\n";
            }
        }
    }   
    close JSTACK;
}
close LOG;

The output helps me to find out which threads are hogging my cpu:

PID     CPU Mem JStack Info
22460   0   8.0 "main" prio=10 tid=0x083cb800 nid=0x57bc runnable [0xb6acc000]
22461   0   8.0 "GC task thread#0 (ParallelGC)" prio=10 tid=0x083d2c00 nid=0x57bd runnable 
22462   0   8.0 "GC task thread#1 (ParallelGC)" prio=10 tid=0x083d4000 nid=0x57be runnable 
22463   0   8.0 "GC task thread#2 (ParallelGC)" prio=10 tid=0x083d5800 nid=0x57bf runnable 
22464   0   8.0 "GC task thread#3 (ParallelGC)" prio=10 tid=0x083d7000 nid=0x57c0 runnable
...

Then I can go back to /tmp/jstack.log and take a look at the stack trace for the problematic thread and try to figure out what's going on from there. Of course this solution is platform-dependent, but it should work with most flavors of *nix and some tweaking here and there.

Cleanly answered 29/3, 2012 at 16:47 Comment(1)

I haven't used this tool much, but it seems to cover my answer and some more: github.com/aragozin/jvm-tools – Cleanly 16/4, 2019 at 16:6

I have created a top-like command specifically for visualizing Java threads ordered by CPU usage and posted the source code at: https://github.com/jasta/jprocps. The command-line syntax is not nearly as rich as top, but it does support some of the same commands:

$ jtop -n 1

Sample output (showing ant and IntelliJ running):

  PID   TID USER       %CPU  %MEM  THREAD
13480 13483 jasta      104   2.3   main
13480 13497 jasta      86.3  2.3   C2 CompilerThread1
13480 13496 jasta      83.0  2.3   C2 CompilerThread0
 4866  4953 jasta      1.0   13.4  AWT-EventQueue-1 12.1.4#IC-129.713, eap:false
 4866 14154 jasta      0.9   13.4  ApplicationImpl pooled thread 36
 4866  5219 jasta      0.8   13.4  JobScheduler pool 5/8

From this output, I can pull up the thread's stack trace in jconsole or jstack manually and figure out what's going on.

NOTE: jtop is written in Python and requires that jstack be installed.

Triolet answered 24/8, 2013 at 21:54 Comment(2)

I get... "from argparse import ArgumentParser ImportError: No module named argparse" – Gingivitis 26/4, 2016 at 19:17

Just prints PID TID USER %CPU %MEM THREAD again and again – Demonography 2/2, 2017 at 9:19

As far as I found out jstack is outdated as of JDK 8. What I used to retrieve all Java Thread names is:

<JDK_HOME>/bin/jcmd <PID> Thread.print

Check jcmd documentation for more.

Demonography answered 2/2, 2017 at 9:36 Comment(1)

If the well-known tmp file .java_pidnnnn already exists for this java process, then you need to run jcmd with the user executing the java process, and it maybe better anyway if the process needs access to this file also. – Sulfite 27/5, 2020 at 10:3

With OpenJDK on Linux, JavaThread names don't propagate to native threads, you cannot see java thread name while inspecting native threads with any tool.

However there is some work in progress:

Personally, I find the OpenJDK development tool slow so I just apply patches myself.

Vole answered 29/3, 2014 at 12:9 Comment(1)

NB: it now works on recent JDK (including recent openjdk8) – Pair 12/7, 2021 at 13:38

Threads don't have names as far as the kernel is concerned; they only have ID numbers. The JVM assigns names to threads, but that's private internal data within the process, which the "top" program can't access (and doesn't know about anyway).

Shindig answered 21/1, 2011 at 22:9 Comment(0)

This shell script combines the output from jstack and top to list Java threads by CPU usage. It expects one argument, the account user that owns the processes.

Name: jstack-top.sh

#!/bin/sh
#
# jstack-top - join jstack and top to show cpu usage, etc.
#
# Usage: jstack-top <user> | view -
#

USER=$1
TOPS="/tmp/jstack-top-1.log"
JSKS="/tmp/jstack-top-2.log"

PIDS="$(ps -u ${USER} --no-headers -o pid:1,cmd:1 | grep 'bin/java' | grep -v 'grep' | cut -d' ' -f1)"
if [ -f ${JSKS} ]; then
    rm ${JSKS}
fi
for PID in ${PIDS}; do
    jstack -l ${PID} | grep "nid=" >>${JSKS}
done

top -u ${USER} -H -b -n 1 | grep "%CPU\|java" | sed -e 's/[[:space:]]*$//' > ${TOPS}
while IFS= read -r TOP; do
    NID=$(echo "${TOP}" | sed -e 's/^[[:space:]]*//' | cut -d' ' -f1)
    if [ "${NID}" = "PID" ]; then
        JSK=""
        TOP="${TOP} JSTACK"
    else
        NID=$(printf 'nid=0x%x' ${NID})
        JSK=$(grep "${NID} " ${JSKS})
    fi
    echo "${TOP}    ${JSK}"
done < "${TOPS}"

Genitalia answered 3/10, 2017 at 16:33 Comment(0)

Expanding on Andre's earlier answer in Perl, here is one in Python that runs significantly faster.

It re-uses files created earlier and does not loop several times over the jstack output:

#!/usr/bin/env python
import re
import sys
import os.path
import subprocess

# Check if jstack.log top.log files are present
if not os.path.exists("jstack.log") or not os.path.exists("top.log"):
  # Delete either file
  os.remove("jstack.log") if os.path.exists("jstack.log") else None
  os.remove("top.log") if os.path.exists("top.log") else None
  # And dump them via a bash run
  cmd = """
  pid=$(ps -e | grep java | sed 's/^[ ]*//g' | cut -d ' ' -f 1)
  top -H -b -n 1 | grep java > top.log
  /usr/intel/pkgs/java/1.8.0.141/bin/jstack -l $pid > jstack.log
  """
  subprocess.call(["bash", "-c", cmd])

# Verify that both files were written
for f in ["jstack.log", "top.log"]:
  if not os.path.exists(f):
    print "ERROR: Failed to create file %s" % f
    sys.exit(1)

# Thread ID parser
jsReg = re.compile('"([^\"]*)".*nid=(0x[0-9a-f]*)')
# Top line parser
topReg = re.compile('^\s*([0-9]*)(\s+[^\s]*){7}\s+([0-9]+)')

# Scan the entire jstack file for matches and put them into a dict
nids = {}
with open("jstack.log", "r") as jstack:
  matches = (jsReg.search(l) for l in jstack if "nid=0x" in l)
  for m in matches:
    nids[m.group(2)] = m.group(1)

# Print header
print "PID\tNID\tCPU\tTHREAD"
# Scan the top output and emit the matches
with open("top.log", "r") as top:
  matches = (topReg.search(l) for l in top)
  for m in matches:
    # Grab the pid, convert to hex and fetch from NIDS
    pid = int(m.group(1))
    nid = "0x%x" % pid
    tname = nids.get(nid, "<MISSING THREAD>")
    # Grab CPU percent
    pct = int(m.group(3))
    # Emit line
    print "%d\t%s\t%d\t%s" % (pid, nid, pct, tname)

Goosy answered 29/1, 2019 at 17:37 Comment(0)

Old question, but I had just the same problem with top.

It turns out, you can scroll top's output to the right simply by using the cursors keys :)

(but unfortunately there won't be any thread name shown)

Barbra answered 7/9, 2015 at 6:11 Comment(0)

You mentioned "Linux". Then using the little tool "threadcpu" might be a solution:

threadcpu_-_show_cpu_usage_of_threads

$ threadcpu -h

threadcpu shows CPU usage of threads in user% and system%

usage:
threadcpu [-h] [-s seconds] [-p path-to-jstack]

options:
  -h display this help page
  -s measuring interval in seconds, default: 10
  -p path to JRE jstack, default: /usr/bin/jstack
example usage:
  threadcpu -s 30 -p /opt/java/bin/jstack 2>/dev/null|sort -n|tail -n 12
output columns:
  user percent <SPACE> system percent <SPACE> PID/NID [ <SPACE> JVM thread name OR (process name) ]

Some sample outputs:

$ threadcpu |sort -n|tail -n 8
3 0 33113 (klzagent)
3 0 38518 (klzagent)
3 0 9874 (BESClient)
3 41 6809 (threadcpu)
3 8 27353 VM Periodic Task Thread
6 0 31913 hybrisHTTP4
21 8 27347 C2 CompilerThread0
50 41 3244 (BESClient)

$ threadcpu |sort -n|tail -n 8
0 20 52358 (threadcpu)
0 40 32 (kswapd0)
2 50 2863 (BESClient)
11 0 31861 Gang worker#0 (Parallel CMS Threads)
11 0 31862 Gang worker#1 (Parallel CMS Threads)
11 0 31863 Gang worker#2 (Parallel CMS Threads)
11 0 31864 Gang worker#3 (Parallel CMS Threads)
47 10 31865 Concurrent Mark-Sweep GC Thread

$ threadcpu |sort -n|tail -n 8
2 0 14311 hybrisHTTP33
2 4 60077 ajp-bio-8009-exec-11609
2 8 30657 (klzagent)
4 0 5661 ajp-bio-8009-exec-11649
11 16 28144 (batchman)
15 20 3485 (BESClient)
21 0 7652 ajp-bio-8009-exec-11655
25 0 7611 ajp-bio-8009-exec-11654

The output is intentionally very simple to make further processing (e.g. for monitoring) more easy.

Schifra answered 19/10, 2018 at 16:24 Comment(0)

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

Recommended topics

Hot tags