Can a finalizer be called on an object that is in the middle of a (native) call?

Question

We're seeing a weird AddressSanitizer (clang/C++) "heap-use-after-free" violation that might relate to a finalizer corner case.

Let's say, a Java object OBJ has a handle to to a native resource X. A thread that created OBJ before, is now making a call on OBJ.method(), which calls into a (static) native method staticMethod(X), in which X is used.

Now, at more or less the same time, we're seeing a thread deleting the native resource X. We strongly assume that this triggered by the finalizer calling OBJ.finalize(), which does "delete X".

Is this a valid thing to do for a finalizer?

(OpenJDK 8)

Rote

Markus, I can't believe how your question have no answers. I'm analyzing the same situation on .NET in this issue but I need an answer for the Java as well. For example in .NET an object can be finalized in the middle of a native call if: 1) there are no more use of the reference of the object after the native call and 2) if the native side invokes execution of managed (read non-native) code, as for example a callback. Could you find a proper answer for your question? – Bjorn

Yes, I was able to fix this meanwhile. See my own answer. – Rote

Answer 1

A safe way to do this seems to be using non-static native JNI methods.

In C/C++, a static JNI method signature looks something like this:

extern "C" JNIEXPORT jobject JNICALL
Java_com_example_MyClass_myMethod(JNIEnv* env, jclass type, jlong handle);

Note the second parameter jclass type passing the JNI representation of a Java class.

A non-static JNI method, however, accepts the current Java instance object (this) instead and looks like this:

extern "C" JNIEXPORT jobject JNICALL
Java_com_example_MyClass_myMethod(JNIEnv* env, jobject thisObj, jlong handle);

Background: The VM seems to optimize garbage collection quite aggressively. A thread still running a (non-static) method but calling only into a native static method does NOT prevent the object from being freed. If the JNI method is non-static however, this tells the VM that the Java object is still being referenced. Then, only once the the call returns this native reference to the object is cleared. Thus, no finalizer is allowed to run before that.

Rote

Thank you! I'm not exactly a Java/JNI developer: I guess the signature changes removing the static modifier from native method in Java. What about the JNI glue code? Can you point me a reference on how is the signature of non-static native JNI method (C/C++ code)? – Bjorn

Answer 2

Default implementation of finalize() method does nothing:

public class Object {
    protected void finalize() throws Throwable { }
}

Your description sounds like shared native resource was removed in one thread while it is required in another thread. You need to check out all native methods (in java) which removes something from native memory space.
Java don't know about objects allocated in native code. You need manually control this via native calls. For example:

public class A {
    private int id;

    static {
        // load native library
    }

    public A(int id) {
        // create required native resources for this instance
        allocateAContext(id)
    }

    // this method will create required native resources out of java heap
    protected native void allocateAContext(int id);

    // this method will remove allocated native resources
    protected native void deleteAContext(int id);

    @Override
    protected void finalize() throws Throwable {
        super.finalize();
        // release native resources when garbage collector will remove A object
        deleteAContext(id);
    }
}

Climb

Thanks Sergey for giving it a shot. I don't understand how your answer relates to my question. This is not about JNI basics, but a very specific corner case. – Rote