Is it possible to track down which expression caused an NPE?

Question

When I get an NPE, I'll get a stack trace with line number. That's helpful, but if the line is very dense and/or contains nested expression, it's still impossible to figure out which reference was null.

Surely, this information must've been available somewhere. Is there a way to figure this out? (If not java expression, then at least the bytecode instruction that caused NPE would be helpful as well)

Edit #1: I've seen a few comments suggesting breaking up the line, etc, which, no offence, is really non-constructive and irrelevant. If I could do that, I would have ! Let just say this modifying the source is out of the question.

Edit #2: apangin has posted an excellent answer below, which I accepted. But it's SOOO COOL that I had to include the output here for anyone who doesn't want to try it out themselves! ;)

So suppose I have this driver program TestNPE.java

 1  public class TestNPE {
 2      public static void main(String[] args) {
 3          int n = 0;
 4          String st = null;
 5  
 6          System.out.println("about to throw NPE");
 7          if (n >= 0 && st.isEmpty()){
 8              System.out.println("empty");
 9          }
10          else {
11              System.out.println("othereise");
12          }
13      }
14      
15  }

The bytecode looks like this (showing only the main() method and omitting other irrelevant parts)

Code:
  stack=2, locals=3, args_size=1
     0: iconst_0
     1: istore_1
     2: aconst_null
     3: astore_2
     4: getstatic     #2                  // Field java/lang/System.out:Ljava/io/PrintStream;                                              
     7: ldc           #3                  // String about to throw NPE                                                                     
     9: invokevirtual #4                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V                                      
    12: iload_1
    13: iflt          34
    16: aload_2
    17: invokevirtual #5                  // Method java/lang/String.isEmpty:()Z                                                           
    20: ifeq          34
    23: getstatic     #2                  // Field java/lang/System.out:Ljava/io/PrintStream;                                              
    26: ldc           #6                  // String empty                                                                                  
    28: invokevirtual #4                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V                                      
    31: goto          42
    34: getstatic     #2                  // Field java/lang/System.out:Ljava/io/PrintStream;                                              
    37: ldc           #7                  // String othereise                                                                              
    39: invokevirtual #4                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V                                      
    42: return

Now when you run the TestNPE driver with the agent, you'll get this

$ java -agentpath:libRichNPE.o TestNPE
about to throw NPE
Exception in thread "main" java.lang.NullPointerException: location=17
    at TestNPE.main(TestNPE.java:7)

So that points to the invokevirtual #5 at offset 17! Just HOW COOL IS THAT?

Frag

Good question, but also keep in mind that if the line is dense the code quality is probably less than optimal. – Masturbation

I know you can't get the column number...but if you can use returns to break the calls on that line out to a series of lines Java will print the relevant row number while still treating it as a line. – Junia

It depends on the JVM implementation. For instance SAP JVM prints the field name which was null in the exception message. – Goulash

Edit to add newlines. – Mauritius

@Svetlin Zarev: There is also a JEP openjdk.java.net/jeps/8220715 that contributes the functionality of SAPJVM to OpenJDK – Larcher

Answer 1

When an exception happens, JVM knows the original bytecode that caused the exception. However, StackTraceElement does not track bytecode indices.

The solution is to capture bytecode index using JVMTI whenever exception occurs.

The following sample JVMTI agent will intercept all exceptions, and if exception type is NullPointerException, the agent will replace its detailMessage with the bytecode location information.

#include <jvmti.h>
#include <stdio.h>

static jclass NullPointerException;
static jfieldID detailMessage;

void JNICALL VMInit(jvmtiEnv* jvmti, JNIEnv* env, jthread thread) {
    jclass localNPE = env->FindClass("java/lang/NullPointerException");
    NullPointerException = (jclass) env->NewGlobalRef(localNPE);

    jclass Throwable = env->FindClass("java/lang/Throwable");
    detailMessage = env->GetFieldID(Throwable, "detailMessage", "Ljava/lang/String;");
}

void JNICALL ExceptionCallback(jvmtiEnv* jvmti, JNIEnv* env, jthread thread,
                               jmethodID method, jlocation location, jobject exception,
                               jmethodID catch_method, jlocation catch_location) {
    if (env->IsInstanceOf(exception, NullPointerException)) {
        char buf[32];
        sprintf(buf, "location=%ld", (long)location);
        env->SetObjectField(exception, detailMessage, env->NewStringUTF(buf));
    }
}

JNIEXPORT jint JNICALL Agent_OnLoad(JavaVM* vm, char* options, void* reserved) {
    jvmtiEnv* jvmti;
    vm->GetEnv((void**)&jvmti, JVMTI_VERSION_1_0);

    jvmtiCapabilities capabilities = {0};
    capabilities.can_generate_exception_events = 1;
    jvmti->AddCapabilities(&capabilities);

    jvmtiEventCallbacks callbacks = {0};
    callbacks.VMInit = VMInit;
    callbacks.Exception = ExceptionCallback;
    jvmti->SetEventCallbacks(&callbacks, sizeof(callbacks));
    jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_VM_INIT, NULL);
    jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_EXCEPTION, NULL);

    return 0;
}

Compile this into a shared library and run java with -agentpath option:

java -agentpath:/pato/to/libRichNPE.so Main

Cesarcesare

Oh, this is interesting! Thanks. I'll give that a try. – Frag

@apngin: Sorry, how do I compile this into a shared library? – Frag

I've got the following error while trying to compile it paste.ubuntu.com/23339517 – Frag

@OneTwoThree This is C++ source. Try g++ -shared -fPIC -I/usr/java/jdk1.8.0_102/include -I/usr/java/jdk1.8.0_102/include/linux -o libRichNPE.so RichNPE.cpp – Cesarcesare

Oh, yes! It compiled! But when I ran it, I'm not seeing anything different: paste.ubuntu.com/23339661 – Frag

@OneTwoThree Yes, there was a mistake in the code. I've fixed the answer. – Cesarcesare

Awesome! Works like a charm. Thank you. This has got to be the coolest trick I've learnt. :) – Frag

PS: Just edited the question to include your solution and output of the run. It's just too cool :) – Frag

Could you please add an example detail message to the answer? I would be intrested in how it actually looks. – Selfness

@Selfness There is an example in the question: java.lang.NullPointerException: location=17 – Cesarcesare

Answer 2

The exception itself does not have enough information to provide more than line numbers.

One option i see is to use a bytecode debugger like bytecode visualizer to closer localize the bytecode instruction that causes the npe. Step forward until the exception occurs, or add a breakpoint for npe.

Prague

Answer 3

JEP 358: Helpful NullPointerExceptions adds such a feature to OpenJDK 14. It is disabled by default; you have to specify -XX:+ShowCodeDetailsInExceptionMessages to enable it. With it, your example results in:

Exception in thread "main"
java.lang.NullPointerException: Cannot invoke "String.isEmpty()" because "st" is null
    at TestNPE.main(TestNPE.java:7)

Classes do not need to be recompiled to take advantage of this feature. It was originally developed for the SAP JVM.

Iila

Answer 4

The stack trace mechanism relies on the debugging metadata optionally compiled into each class (namely the SourceFile and LineNumberTable attributes). As far as I know, bytecode offsets are not preserved anywhere. However, these would not be useful for a typical Java program, since you still have know what code each bytecode instruction corresponds to.

However, there is an obvious workaround - just break the code in question up into multiple lines and recompile! You can insert whitespace almost anywhere in Java.

Loy

Answer 5

You can either break up the complex line into many smaller ones you can trace, or you use your debugger to see what value was null when the exception occurred.

While you could try to look at the byte code where this happened, this will only be the start of a complex journey. I suggest making your code simpler to understand and you might work out which values could be null (Note: it could be null unless you know it's not possible)

Hurtado