How do I analyze a .hprof file?

Question

I have a production server running with the following flag: -XX:+HeapDumpOnOutOfMemoryError

Last night it generated a java-38942.hprof file when our server encountered a heap error. It turns out that the developers of the system knew of the flag but no way to get any useful information from it.

Any ideas?

Gushy

Answer 1

If you want a fairly advanced tool to do some serious poking around, look at the Memory Analyzer project at Eclipse, contributed to them by SAP.

Some of what you can do is mind-blowingly good for finding memory leaks etc -- including running a form of limited SQL (OQL) against the in-memory objects, i.e.

SELECT toString(firstName) FROM com.yourcompany.somepackage.User

Totally brilliant.

Undis

I'd just like to add a +100 for Eclipse Memory Analyzer. I'm currently attempting to sift thru a 400mb+ heap dump file, and it took jhat more than 70 minutes to read the file, before it caused a complete JVM crash. EMA is able to open it up in < 5 minutes. – Slough

I keep getting parsing errors when opening HPROF files using the Eclipse Memory Analyzer (that were in fact also dumped by Eclipse!). Unfortunate.. sigh. – Bicycle

MAT can still require quite a bit of RAM [less than JHAT but still quite a bit]. See #7254517 for tips if you run into that happening. – Kort

Error opening heap dump 'strictmode-death-penalty.hprof'. Check the error log for further details. Error opening heap dump 'strictmode-death-penalty.hprof'. Check the error log for further details. Unknown HPROF Version (JAVA PROFILE 1.0.3) (java.io.IOException) Unknown HPROF Version (JAVA PROFILE 1.0.3) – Irisirisa

Answer 2

You can use JHAT, The Java Heap Analysis Tool provided by default with the JDK. It's command line but starts a web server/browser you use to examine the memory. Not the most user friendly, but at least it's already installed most places you'll go. A very useful view is the "heap histogram" link at the very bottom.

ex: jhat -port 7401 -J-Xmx4G dump.hprof

jhat can execute OQL "these days" as well (bottom link "execute OQL")

Devilment

After running above command In the console you will get the message int the terminal "server is ready" Port : 7401. After that open this URL : localhost:7401 "you can see details in the browser windows ". – Paucker

java.io.IOException: Version string not recognized at byte 17 – Irisirisa

This is not delivered by default in the latest versions of OpenJDK or Oracle JDK. – Spiral

Answer 3

You can also use HeapWalker from the Netbeans Profiler or the Visual VM stand-alone tool. Visual VM is a good alternative to JHAT as it is stand alone, but is much easier to use than JHAT.

You need Java 6+ to fully use Visual VM.

Goldina

You should add a note that this is for Java 6 and 7 only. – Gushy

AFAIK, HeapWalker and VisualVM do not require Java 6/7 to read HPROF files. – Goldina

I just tried to load it with Java 5 and it said 'Please use Java 6 or 7' What am I doing wrong? – Gushy

Hmm. Perhaps you have to run Java 6, but can read a Java 5 HPROF (maybe that's how I got it to work). I know you cannot create a heap-dump from Visual VM on an app running Java 5. Will update. – Goldina

You can also use visual JVM, from java SDK - look for jvisualvm.exe at JAVASDK/bin folder – Val

Answer 4

Just get the Eclipse Memory Analyzer. There's nothing better out there and it's free.

JHAT is only usable for "toy applications"

Bearden

JHAT is necessary for impressing those "l33t" hackers who hand-build a BSD distro starting with the LILO. Wait... they'd never use java anyway. :-) – Goldina

I think this is more of a comment... :\ – Kort

Yeah, JHAT is for "toy applications". I had a dump file (9GB), JHAT runs about 30 mins with 20GB heap and fails with OOM finally. The Eclipse Memory Analyzer handles it with 15GB heap within 1 min. – Autoerotic

when I open a 3 GB dump file, The Eclipse Memory Analyzer, show out of memory error and unable to open the dump file. Windows 10/JRE17 - An internal error occurred during: "Parsing heap dump from 'D:\java_pid12269.hprof'". java.lang.OutOfMemoryError – Gisela

Answer 5

I personally prefer VisualVM. One of the features I like in VisualVM is heap dump comparison. When you are doing a heap dump analysis there are various ways to go about figuring out what caused the crash. One of the ways I have found useful is doing a comparison of healthy vs unhealthy heap dumps.

Following are the steps you can follow for it :

1. Getting a heap dump of OutOfMemoryError let's call it "oome.hprof". You can get this via JVM parameter HeapDumpOnOutOfMemoryError.
2. Restart the application let it run for a bit (minutes/hours) depending on your application. Get another heap dump while the application is still running. Let's call it "healthy.hprof".
3. You can open both these dumps in VisualVM and do a heap dump comparison. You can do it on class or package level. This can often point you into the direction of the issue.

link : https://visualvm.github.io

Mescaline

Answer 6

YourKit Java Profiler seems to handle them too.

Caviar

Answer 7

If you want to do a custom analysis of your heapdump then there's:

• JVM Heap Dump Analysis library https://github.com/aragozin/heaplib/tree/master/hprof-heap

This library is fast but you will need to write your analysis code in Java.

From the docs:

• Does not create any temporary files on disk to process heap dump
• Can work directly GZ compressed heap dumps
• HeapPath notation

Civilized

Link is no more active. – Birchfield

Answer 8

I'm using IntelliJ IDEA profiler in Ultimate version. In IDEA, press the Shift button twice.
In Actions tab write - "profiler".
Choice row "profiler" with icon.
Then Open snapshot.. button.

Stillage