Deallocating Direct Buffer Native Memory in Java for JOGL

Question

I am using direct buffers (java.nio) to store vertex information for JOGL. These buffers are large, and they are replaced several times during the application life. The memory is not deallocated in time and I am running out of memory after a few replacements.

It seems that there is not good way to deallocate using java.nio's buffer classes. My question is this:

Is there some method in JOGL to delete Direct Buffers? I am looking into glDeleteBuffer(), but it seems like this only deletes the buffer from the video card memory.

Thanks

Turcotte

Answer 1

The direct NIO buffers use unmanaged memory. It means that they are allocated on the native heap, not on the Java heap. As a consequence, they are freed only when the JVM runs out of memory on the Java heap, not on the native heap. In other terms, it's unmanaged = it's up to you to manage them. Forcing the garbage collection is discouraged and won't solve this problem most of the time.

When you know that a direct NIO buffer has become useless for you, you have to release its native memory by using its sun.misc.Cleaner (StaxMan is right) and call clean() (except with Apache Harmony), call free() (with Apache Harmony) or use a better public API to do that (maybe in Java > 12, AutoCleaning that extends AutoCloseable?).

It's not JOGL job to do that, you can use plain Java code to do it yourself. My example is under GPL v2 and this example is under a more permissive license.

Edit.: My latest example works even with Java 1.9 and supports OpenJDK, Oracle Java, Sun Java, Apache Harmony, GNU Classpath and Android. You might have to remove some syntactical sugar to make it work with Java < 1.7 (the multi catches, the diamonds and the generics).

Reference: http://www.ibm.com/developerworks/library/j-nativememory-linux/

Direct ByteBuffer objects clean up their native buffers automatically but can only do so as part of Java heap GC — so they do not automatically respond to pressure on the native heap. GC occurs only when the Java heap becomes so full it can't service a heap-allocation request or if the Java application explicitly requests it (not recommended because it causes performance problems).

Reference: http://docs.oracle.com/javase/7/docs/api/java/nio/ByteBuffer.html#direct

The contents of direct buffers may reside outside of the normal garbage-collected heap

This solution is integrated in Java 14:

try (MemorySegment segment = MemorySegment.allocateNative(100)) {
   ...
}

You can wrap a byte buffer into a memory segment by calling MemorySegment.ofByteBuffer(ByteBuffer), get its memory address and free it (this is a restricted method) in Java 16:

CLinker.getInstance().freeMemoryRestricted(MemorySegment.ofByteBuffer(myByteBuffer).address());

Note that you still need to use reflection in many non trivial cases in order to find the buffer that can be deallocated, typically when your direct NIO buffer isn't a ByteBuffer.

N.B: sun.misc.Cleaner has been moved into jdk.internal.ref.Cleaner in Java 1.9 in the module "java.base", the latter implemented java.lang.Runnable (thanks to Alan Bateman for reminding me that difference) for a short time but it's no longer the case. You have to call sun.misc.Unsafe.invokeCleaner(), it's done in JogAmp's Gluegen. I preferred using the Cleaner as a Runnable as it avoided to rely on sun.misc.Unsafe but it doesn't work now.

My last suggestion works with Java 9, 10, 11 and 12.

My very latest example requires the use of an incubated feature (requires Java >= 14) but is very very simple.

N.B: Using a memory segment to free a preexisting direct NIO buffer is no longer possible since Java 18. Instead, create a native memory segment by calling MemorySegment.allocateNative(), call asByteBuffer() on it and close its memory session when you no longer need it. It's a lot more invasive but there are probably excellent API design decisions behind this change. Maybe MemorySegment.ofBuffer() in Java >= 21 helps but I fear that it uses a global arena that you can't safely close.

There is a good example in Lucene under a more permissive license.

Landowska

+1: This is in fact not related to JOGL and the best reference I could find here on releasing java nio buffers – Hole

The last solution on your list probably won't be ready for Java 9. The JEP is still a draft. openjdk.java.net/jeps/191 – Shiflett

sun.misc.unsafe has been renamed to jdk.internal.misc.Unsafe in recent JDK versions (although there is still a shim, for now at least, at sun.misc.Unsafe that proxies calls to jdk.internal.misc.Unsafe). See: github.com/classgraph/classgraph/blob/master/src/main/java/… – Deepfreeze

@LukeHutchison sun.misc.Unsafe contains the method named invokeCleaner(java.nio.ByteBuffer) whereas jdk.internal.misc.Unsafe doesn't. Therefore, it's not exactly a renaming. – Landowska

Thanks for pointing that out -- this is probably because the JDK team intends to remove the cleaner mechanism at some point, replacing it with a newer alternative. – Deepfreeze

One thing I noticed: in JDK9+, calling sun.misc.Cleaner.clean() gives a reflection (encapsulation) warning on stderr. Unsafe.theUnsafe.invokeCleaner(byteBuffer) makes the same call, but does not print the reflection warning. – Deepfreeze

@LukeHutchison Thank you for this finding, I hadn't found this one. I advise you to use MemorySegment.ofByteBuffer(ByteBuffer).close() when the cleaner mechanism doesn't work. – Landowska

@Landowska I tried adding that call to my code via reflection. However, since the feature is still in preview, this requires a commandline switch. Also in a modular classpath, you have to use "requires" in the module descriptor with MemorySegment's current (temporary) module. You can't even compile the module descriptor without enabling preview features. Best to wait until this feature is mainlined, I think. – Deepfreeze

Other issues (I didn't check if your code handles all of these): (1) Attempting to clean the same DirectByteBuffer twice results in the VM crashing. (2) Attempting to clean a ByteBuffer that is not a DirectByteBuffer throws IllegalArgumentException. (3) Attempting to clean ByteBuffer that is a slice or duplicate of an underlying DirectByteBuffer also fails (I think also with IllegalArgumentException?). My code handles these: github.com/classgraph/classgraph/blob/master/src/main/java/… – Deepfreeze

@LukeHutchison (1) I didn't reproduce the problem. (2) I already test whether the buffer is direct. (3) I handle and test this case for sure: sourceforge.net/p/tuer/code/HEAD/tree/pre_beta/src/test/java/… I advise you to use similar tests. My source code (under GPL) is far from being trivial. – Landowska

@Landowska Sorry, I should have clarified: the JVM crash can happen with (1) if you unmap (clean) a MappedByteBuffer (which extends DirectByteBuffer), and then subsequently try to access the unmapped buffer. See here: mapdb.org/blog/mmap_files_alloc_and_jvm_crash Your code works well, I'm sure, but the supported runtimes are overkill for my usecase. Also I can't use the GPL for my projects, because the virality of the license is too restrictive. I used to be 100% in support of the GPL, but now I see it causes more problems than it solves. I use the MIT license for everything. – Deepfreeze

@LukeHutchison Thank you for the clarification. There is nothing I can do yet, it's up to the caller not to keep a strong reference on a cleaned byte buffer. Yes, you're right, I keep it online mostly for pedagogical purposes because it will be overkill for my use case soon too because my first person shooter uses OpenJDK 14. I won't debate about licenses here because I use the GPL for political reasons, each developer has his own opinions on intellectual property, capitalism, the coexistence of free software and proprietary software, etc. – Landowska

@Landowska Fair enough -- yes, it's a personal preference thing. I do appreciate that you are sharing your knowledge on this issue. I wish the JDK team had moved on this issue a long time ago. – Deepfreeze

@LukeHutchison You only need to use attachment() when you use MemorySegment.close(), it's a lot easier now but I still get a warning. – Landowska

Answer 2

Direct buffers are tricky and don't have the usual garbage collection guarantees - see for more detail: http://docs.oracle.com/javase/7/docs/api/java/nio/ByteBuffer.html#direct

If you are having issues, I'd suggest allocating once and re-using the buffer rather than allocating and deallocating repeatedly.

Schargel

Your last suggestion is quite good. The buffers are really useful when they are used more than once. It's possible to slice them too. However, there is a risk of increasing the memory footprint at runtime by keeping unused allocated native memory. When it is implemented correctly, the deallocation of direct NIO buffers is more efficient and less invasive than trying to avoid allocating and deallocating them repeatedly. – Landowska

I actually dramatically improved memory management one of my apps that uses a lot of direct I/O buffers by doing this - I just cache the buffers and reuse them as needed, rather than try to release them back to the OS. Of course, the app is now a memory pig after a large data set, but that's not a concern for me - it's running on a more or less dedicated host. – Vegetarianism

Answer 3

The way in which deallocation is done is awful -- a soft reference is basically inserted into a Cleaner object, which then does deallocation when owning ByteBuffer is garbage collected. But this is not really guaranteed to be called in timely manner.

Obreption

You call it awful, but AFAIK it's as good as you can get it. Direct buffers are intentionally allocated off the managed heap so we shouldn't be surprised by their management being a bit lacking. Anyway, before an OOM gets thrown, there are attempts to process all pending references and the GC gets run. It mayn't work as designed. +++ I'm afraid, that's all we can get unless we resolve to Unsafe and risk malloc/free-like leaks and crashes. – Selfpropelled

Answer 4

Deallocating a Direct Buffer is a job done by the garbage collector some time after the ByteBuffer object is marked.

You could try calling the gc immediatly after deleting the last reference to your buffer. At least there's a chance that the memory will be free'd a bit faster.

Tenedos

By "calling the gc" you mean System.gc() I presume; it is indeed a long shot, as that call isn't guaranteed to really do anything. Make sure to set any reference to the direct buffer = null before calling that, or the GC will definitely think the buffer is still in use. – Audry

Answer 5

Using information in gouessej's answer, I was able to put this utility class together for freeing the direct memory allocation of a given ByteBuffer. This, of course, should only be used as a last resort, and shouldn't really be used in production code.

Tested and working in Java SE version 10.0.2.

public final class BufferUtil {

    //various buffer creation utility functions etc. etc.

    protected static final sun.misc.Unsafe unsafe = AccessController.doPrivileged(new PrivilegedAction<sun.misc.Unsafe>() {
        @Override
        public Unsafe run() {
            try {
                Field f = Unsafe.class.getDeclaredField("theUnsafe");
                f.setAccessible(true);
                return (Unsafe) f.get(null);
            } catch(NoSuchFieldException | SecurityException | IllegalArgumentException | IllegalAccessException ex) {
                throw new RuntimeException(ex);
            }
        }
    });

    /** Frees the specified buffer's direct memory allocation.<br>
     * The buffer should not be used after calling this method; you should
     * instead allow it to be garbage-collected by removing all references of it
     * from your program.
     * 
     * @param directBuffer The direct buffer whose memory allocation will be
     *            freed
     * @return Whether or not the memory allocation was freed */
    public static final boolean freeDirectBufferMemory(ByteBuffer directBuffer) {
        if(!directBuffer.isDirect()) {
            return false;
        }
        try {
            unsafe.invokeCleaner(directBuffer);
            return true;
        } catch(IllegalArgumentException ex) {
            ex.printStackTrace();
            return false;
        }
    }

}

Ras

Answer 6

Rather than abuse reflection of non-public apis you can do this trivially, entirely within the supported public ones.

Write some JNI which wraps malloc with NewDirectByteBuffer (remember to set the order), and an analogous function to free it.

Andonis

I agree with ochi and look at the very last suggestion in my post which doesn't use any non public APIs. – Landowska