How to determine in java when I'm near to the OutOfMemoryError?

Question

I need to find out when I'm really close to the OutOfMemoryError so I can flush results to file and call runtime.gc();. My code is something like this:

Runtime runtime = Runtime.getRuntime();
...
if ((1.0 * runtime.totalMemory() / runtime.maxMemory()) > 0.9) {
... flush results to file ...
  runtime.gc();
}

Is there a better way to do this? Can someone give me a hand please?

EDIT

I understood that I am playing with fire this way so I reasoned to a more solid and simple way of determining when I've had enough. I am currently working with the Jena model so I do a simple check: if the model has more than 550k statements then I flush so I don't run any risks.

Leadbelly

If you want your program to fail miserably and mysteriously at random times which cannot be correlated with any outside factor, I warmly recommend you to build this into your project. – Food

This is terrible! How about you fix the source of the OutOfMemoryError instead? – Uvea

I was thinking to do a try/catch of the OutOfMemoryError and only then flush everything to file and call the garbage collector but still I do not want to fire the error. I just want to know when I am really near. – Leadbelly

OOME is typically thrown as a result of the garbage collector finding that it cannot reclaim enough space to satisfy an allocation request. How you are planning to get more space if the JVM can't? – Pocketful

@Pocketful I flush the results to file, dereference the variable and create new ones. – Leadbelly

That is just so wrong. There is only one proper design: prevention, not reaction. If your application runs out of memory, tune it to use less memory at the same moment in time. Like flushing sooner rather than later. – Hambley

Like I said, how do I know when to flush? I need to know when I'm near the OOME. – Leadbelly

Answer 1

First: if you want to determine if you're close to OutOfMemoryError, then what all you have to do is to compare the current memory with the max memory used by JVM, and that what you already did.

Second: You want to flush results to file, am wondering why you want to do that just if you close to OutOfMemoryError, you simply can use something like a FileWriter which has a buffer, so if the buffer got filled it will flush the results automatically.

Third: don't ever call the GC explicitly, its a bad practice, optimize your JVM memory arguments instead:

-Xmx -> this param to set the max memory that the JVM can allocate
-Xms -> the init memory that JVM will allocate on the start up
-XX:MaxPermSize= -> this for the max Permanent Generation memory

Also

-XX:MaxNewSize=  -> this need to be 40% from your Xmx value
-XX:NewSize= -> this need to be 40% from your Xmx value

These will speed up the GC.

And -XX:+UseConcMarkSweepGC to enable using CMS for the old space.

Laborious

I cannot use FileWriter because I am using the Jena model to write to file so I must write an entire file at a time. I think if I increase the heap size to 1GB it will do. – Leadbelly

You can flush periodically, i don't know how you generate your results but i believe you can do flush before get that memory leak. – Laborious

And that's what I'm trying to avoid: if I know when I'm near to OOME then I can flush and avoid it. – Leadbelly

@Ariel, what am trying to say that you don't have to be close with OutOfMemoryError at all, its not a healthy way, you can depends on something else. – Laborious

Answer 2

This seems to work:

public class LowMemoryDetector {

    // Use a soft reference to some memory - will be held onto until GC is nearly out of memory.
    private final SoftReference<byte[]> buffer;
    // The queue that watches for the buffer to be discarded.
    private final ReferenceQueue<byte[]> queue = new ReferenceQueue<>();
    // Have we seen the low condition?
    private boolean seenLow = false;

    public LowMemoryDetector(int bufferSize) {
        // Make my buffer and add register the queue for it to be discarded to.
        buffer = new SoftReference(new byte[bufferSize], queue);
    }

    /**
     * Please be sure to create a new LMD after it returns true.
     * 
     * @return true if a memory low condition has been detected.
     */
    public boolean low () {
        // Preserve that fact that we've seen a low.
        seenLow |= queue.poll() != null;
        return seenLow;
    }
}

private static final int OneMeg = 0x100000;

public void test() {
    LowMemoryDetector lmd = new LowMemoryDetector(2*OneMeg);
    ArrayList<char[]> eatMemory = new ArrayList<>();
    int ate = 0;
    while ( !lmd.low() ) {
        eatMemory.add(new char[OneMeg]);
        ate += 1;
    }
    // Let it go.
    eatMemory = null;
    System.out.println("Ate "+ate);
}

it prints

Ate 1070

for me.

Use a buffer size of something larger than the largest allocation unit you are using. It needs to be big enough so that any allocation request would be satisfied if the buffer was freed.

Please remember that on a 64bit JVM it is potentially possible that you are running with many tb of memory. This approach would almost certainly encounter many difficulties in this case.

Importune