Monitoring java native memory

Question

We are monitoring jvm metrics like heap, metaspace, threads and gc count and we are able to push these metrics to monitorng server like prometheus. Similarly we wanted to track Java native memory metrics(output of jcmd VM.sumary). My question is, is it possible to get these metrics by calling any jvm runtime classes?

Alleyway

Monitoring VM.sumary is not really useful, since NMT may often report less memory than the actual usage, as well as it may report more than the real RSS of the process. I'd suggest to monitor RSS instead. – Outrage

On Linux, monitoring RSS from is very easy - just parse /proc/[pid]/stat output. To monitor the JVM native memory usage from within the application, parse /proc/self/stat. – Outrage

Thanks @Outrage for the links. My whole idea is to decrease the process memory, but i want to monitor native memory usage so that we get an idea of where most of our memory is going. – Alleyway

@UdayShankar, if you're into parsing the /proc/... files than you can look at /proc/</pid>/maps too. But as Shipilev says (shipilev.net/jvm/anatomy-quarks/12-native-memory-tracking): "This [Using JVM's NMT] is much, much, much easier than trying to figure out what JVM is doing by parsing the opaque memory maps from e.g. /proc/$pid/maps" – Myth

Answer 1

Yes, it is possible to get NativeMemoryTracking summary directly from a Java application:

import javax.management.JMException;
import javax.management.ObjectName;
import java.lang.management.ManagementFactory;

public class DiagnosticCommand {

    public static String execute(String command, String... args) throws JMException {
        return (String) ManagementFactory.getPlatformMBeanServer().invoke(
                new ObjectName("com.sun.management:type=DiagnosticCommand"),
                command,
                new Object[]{args},
                new String[]{"[Ljava.lang.String;"});
    }

    public static void main(String[] args) throws Exception {
        String summary = DiagnosticCommand.execute("vmNativeMemory", "summary");
        System.out.println(summary);
    }
}

You'll have to parse the text output though.

Note that the most important parts of the NMT report can be tracked separately with the designated MBeans, including

• Java Heap
• Code Cache
• Metaspace
• Compressed Class Space
• Direct Byte Buffers and Mapped Byte Buffers

See MemoryPoolMXBean and BufferPoolMXBean.

As I told in the comments, monitoring NMT output is not always helpful in practice, since it does not directly reflect the actual physical memory used by the process. NMT can report much less memory than the actual usage, or it can also report more memory than the process consumes from the OS perspective.

Since NMT can miss the large amount of OS memory consumed by a Java process, it's also useful to monitor the process' resident set size (RSS). On Linux this can be done by parsing /proc/[pid]/stat or /proc/[pid]/status.

Outrage

Answer 2

I don't think there's a Java API for that. Your best bet might be to invoke the jcmd <PID> VM.native_memory command and parse its output. Of course, you need to enable native memory tracking for your process first.

Myth

There is a Java API for that. Though it still requires parsing the text output. – Outrage

Answer 3

You can find many things you want in JMX. One of the answers to question "Why Use JMX Technology?" is "Monitor and manage the Java VM". In Oracle doc:

The Java Virtual Machine (Java VM) is highly instrumented using the JMX technology. You can start a JMX agent to access the built-in Java VM instrumentation, and thereby monitor and manage a Java VM remotely.

//Metaspace
for (MemoryPoolMXBean memoryMXBean : ManagementFactory.getMemoryPoolMXBeans()) {
    if ("Metaspace".equals(memoryMXBean.getName())) {
            System.out.println(memoryMXBean.getUsage().getUsed());
            System.out.println(memoryMXBean.getUsage().getCommitted());
            System.out.println(memoryMXBean.getUsage().getMax());
            System.out.println(memoryMXBean.getUsage().getInit());
    }
}

//current number of live threads including both daemon and non-daemon threads
int threadCount = ManagementFactory.getThreadMXBean().getThreadCount();

//Returns the current memory usage of the heap and non-heap
MemoryMXBean memoryMXBean = ManagementFactory.getMemoryMXBean();
MemoryUsage heapMemory = memoryMXBean.getHeapMemoryUsage();
MemoryUsage nonHeapMemory = memoryMXBean.getNonHeapMemoryUsage();

//GarbageCollector total number of collections
List<GarbageCollectorMXBean> garbageCollectorMXBeans = ManagementFactory.getGarbageCollectorMXBeans();
long totalCollectionCount = garbageCollectorMXBeans.stream().mapToLong(x -> x.getCollectionCount()).sum();

Weld

Thanks for the answer, but I am more interested in finding out exactly how other jvm memory parts like Code, GC, Compiler, Internal, Symbol and Unknown parts are increasing as the process times – Alleyway

Answer 4

Allthough you don't mention Micrometer, I'd like to point out a little support library I have written for getting ahold of the procfs information like Resident Set Size (RSS). Perhaps you can extract some useful code out of it. Actually the procfs-code is not Micrometer-related (allthough not considered public API). Here it is: https://github.com/mweirauch/micrometer-jvm-extras

Donohue

thanks @Donohue for the github project, i can use that for getting process overall memory. I am also interested in finding out exactly how other jvm memory parts like Code, GC, Compiler, Internal, Symbol and Unknown parts are increasing as the process times. – Alleyway