Does G1GC release back memory to the OS even if Xms = Xmx?
Asked Answered
I

1

9

After reading some answers like this and JEP-346, I have realised that the G1 does release memory back to the OS.

However does it release memory back to the OS, even to the point that current memory use can drop below the initial heap memory (i.e before this JEP, in my case JDK11)?

Assume I have a Java 11 VM running with Xms and Xmx set as 5GB, on a 8GB RAM, however I am consuming only around 1GB. Will G1 release enough memory back to the OS?

I didn't find any documentation anywhere which says that the G1 is restricted to releasing keeping the Xms threshold in mind.

I am observing this in Production, the MemAvailable keeps on decreasing till a point, and then after a GC, it jumps upto close to 30-35% on an 8GB box. So I am assuming it is releasing memory, which is why the MemAvailable is jumping back up.

Also what does it exactly mean to release memory to the OS, is it calling free/unmap ?

Irrefragable answered 16/12, 2019 at 18:58 Comment(0)
P
20

Note: I have deleted my previous answer and have studied the sources (also build my own JVM just to find out this particular moment), here is the answer.

The short answer

JVM 11 version (at the moment), will not go below Xms when making the heap smaller.

The long answer

The absolute truth is in the source code. And here is the decision taken to shrink the heap or not. A few lines below, you can see that if we enter that if, there will be a log statement:

Attempt heap shrinking (capacity higher than max desired capacity after Full GC).

So in essence, if we can understand two parameters : capacity_after_gc and maximum_desired_capacity - we can solve this mystery. In general, capacity_after_gc is not something easy to grasp; mainly because it depends on how much garbage there was and how much the current GC could reclaim. For simplicity, I am going to write some code that does not generated any garbage, so that this value is constant.

In such a case, we only need to understand maximum_desired_capacity.

A few lines above, you can see that this is computed as :

maximum_desired_capacity =  MAX2(maximum_desired_capacity, min_heap_size);

Unfortunately, this is where it gets tricky, because it's a lot of code to follow and understand to really see how these ergonomics are set; especially since they depend on various arguments that the JVM has been started with.

For example min_heap_size is set as:

// If the minimum heap size has not been set (via -Xms),
// synchronize with InitialHeapSize to avoid errors with the default value.

Notice, that they even refer to -Xms as minimum; though the documentation says it's initial. You can also notice that it further depends on another two properties :

 reasonable_minimum , InitialHeapSize

This will be difficult to explain further; that is why I will not. Instead, I will show you some simple proof (I did go through the majority of that code...)


Suppose you have this very simple code:

public class HeapShrinkExpand {

    public static void main(String[] args) throws InterruptedException {

        for (int i = 0; i < 10; i++) {
            Thread.sleep(500);
            System.gc();
        }
    }
}

And I run it with:

-Xmx22g 
-XX:InitialHeapSize=1g
"-Xlog:heap*=debug" 
"-Xlog:gc*=debug" 
"-Xlog:ergo*=debug" 

In logs, I will see:

[0.718s][debug][gc,ergo,heap   ] GC(0) Attempt heap shrinking (capacity higher than max desired capacity after Full GC). Capacity: 1073741824B occupancy: 8388608B live: 1018816B maximum_desired_capacity: 27962026B (70 %)
[0.719s][debug][gc,ergo,heap   ] GC(0) Shrink the heap. requested shrinking amount: 1045779798B aligned shrinking amount: 1044381696B attempted shrinking amount: 1044381696B

This tells you some stats around how much shrinking is desired, what is the current capacity, etc. The next line will show you how much the heap has gone down, actually:

[0.736s][debug][gc,ihop] GC(0) Target occupancy update: old: 1073741824B, new: 29360128B

The heap did shrink, down to around 29MB.

If I add a single JVM start-up flag: -Xms10g, those GC logs that are responsible for showing how much the heap was shrank to; will not be present anymore.

And in fact if I run my own JMV (with some logging enabled), those two values: capacity_after_gc and maximum_desired_capacity will always have the same values; meaning that if statement will never be entered and heap will never go below -Xms.


I have run the same code with JDK-13 and, while the shrinking logs are present there (when -Xms is given as an argument), the underlying heap stays at the -Xms, still. What I find even more interesting is that under java-13, trying to run with:

 -Xmx22g -Xms5g -XX:InitialHeapSize=1g

will correctly error out with:

Incompatible minimum and initial heap sizes specified

Prothonotary answered 17/12, 2019 at 15:10 Comment(8)
The JEP is explicit about it, but my question was is this the current status quo in JDK11 too? Edited my question to make this more explicit.Irrefragable
@AdwaitKumar AFAIK yes, but when exactly this will happen - you will need to study the G1 source code. people say that it does and I believe them.Prothonotary
I posted a comment on the above answer and got back no as a reply.Irrefragable
@AdwaitKumar I commented too in there, it seems the other person does not really understand things either; that makes me even more correct now.Prothonotary
thanks for looking into it. Is it possible for me to verify it myself via a small program? I wrote one to create memory pressure and analyzed GC logs, but couldn't find indication of heap resizing. Perhaps I am not looking at the right things.Irrefragable
@AdwaitKumar sorry for the long delay, I edited the answer.Prothonotary
Beautifully explained @Eugene, worth the wait. Thanks.Irrefragable
Good one @ProthonotaryVersicular

© 2022 - 2024 — McMap. All rights reserved.