Can Java write to / read from off heap memory that was freed?

Question

This did surprise me, I'm playing with Java Unsafe. Basically what I am testing is

Allocate unsafe memory -> free the memory -> Write to the freed memory

I was expecting to see some kind of segmentation fault error as I am accessing memory that was freed, but surprisingly, no error / exception was raised.

My code is:

protected static final Unsafe UNSAFE;
static {
    try {
        Field field = Unsafe.class.getDeclaredField("theUnsafe");
        field.setAccessible(true);
        UNSAFE = (Unsafe) field.get(null);
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}
@Test
public void test() {
    long ptr = UNSAFE.allocateMemory(1000);
    UNSAFE.freeMemory(ptr);
    UNSAFE.putOrderedLong(null, ptr, 100L);
}

My question is, if so then why do we need the freeMemory() function in Unsafe? What is it really for?

Stultz

Well if it is anything like C and C++ then you can (doesn't mean you should) write to freed memory as much as you want. It is undefined behavior. It will probably crash (eventually) but a simple int*p = malloc(1000); free(p); p[0] = 1; will "work" (by work I mean not crash) (as will the same using new and delete.) – Aggie

Which operating system are you running this on? – Valerio

@ElliottFrisch I am testing on my Mac – Stultz

@Aggie I am thinking about a theory but not sure if that is a right way of thinking: The memory space of machine is fixed, and each time a new JVM is started, it would need to use some memory. So I am assuming allocateMemory is to mark the memory as occupied, and then freeMemory is to unmark them so that OS can allocate the memory to other processes. But in the event the freed memory is not reallocated to other processes, it is still accessible to everyone? – Stultz

AFAIK in C, what malloc and free do is to help you manage the memory to prevent memory leak. malloc return pointer to memory address that is not allocated by malloc before, or allocated by malloc then free before. If you do not free an address allocated by malloc when you no longer need that memory, that address will then be not usable in the future. If you free the address, and write to it, malloc may consider that address is free and allocate it to subsequent malloc call, which may cause your data at that adress lost. Not sure if this is the same in java. – Catadromous

Almost certainly you will never be able to access any memory that has been allocated to another process. Unsafe is not controlling the physical RAM chips of your computer---that's what operating systems are for. The memory that Unsafe is unsafely allocating and deallocating for you is virtual memory that the operating system allocates to your process, either in advance or upon demand. – Cressida

Check the malloc man page for OS X, and there are many options that may or may not be set. MallocScribble is particularly apropos: If set, fill memory that has been deallocated with 0x55 bytes. This increases the likelihood that a program will fail due to accessing memory that is no longer allocated. – Valerio

Answer 1

You only get a signal if you either read addresses which hasn't been assigned a page or write to memory which is read only or not assigned a page.

When you allocate small regions of memory it takes it from a native memory pool which is generally not released back to the operating system when freed.

If you free memory and then allocate it later, that memory can be used for a different purpose. Continuing to use the old address can lead to corruption.

Hemistich

this is actually only what malloc and free will do on a particular OS, isn't it? – Pew

@Pew correct. A specific JVM could use a different library but I think that would be surprising. – Hemistich

than this means that you might get a segmentation fault wrapped into some java exception or it's really undefined what will happen, right? – Pew

@Pew Java never wraps a SIGSEG or SIGBUS error. It kills the JVM. What I do is have an accessor object with the off-heap address. I clear the accessor object to null so that I am likely to get an NPE instead. (Not guaranteed as I don't make this thread safe) – Hemistich

Answer 2

This might be totally off, but I'll post this anyways.

It seems to me that this is similar to how it works in C (as @John3136 suggested). Looking at the "official" documentation (http://www.docjar.com/html/api/sun/misc/Unsafe.java.html), it's pretty clear that the memory you allocated is native memory. That means that it's in the JVM memory but in the heap allocated to the JVM by the OS itself.

When the OS allocated a chunk of memory, it allocates a certain address range to this memory, and (apart from a couple of other things like Static memory and code) the heap grows from the bottom, and the stack from the top of the address space (hopefully this is familiar). So if there is memory allocated in the heap, and you reference it again, there is no possible way that you are stepping outside the bounds allocated by the OS. In fact, I'm sure if you try to do UNSAFE.putOrderedLong(null, ptr+4, 100L) it's still going to work if your computer is new enough. Now what's stored exactly in there is a mystery even to the OS itself, but since UNSAFE operates un such a low level, it's definitely possible.

Scour

What do you mean by "That means that it's in the JVM memory but in the heap allocated to the JVM by the OS itself" ? Unsafe is manipulating memory outside heap and that's why it is called off-heap, correct? – Stultz

@Stultz Based on the documentation I've read, and this is how I think any program running on the OS operates, it operates on the heap allocated to the JVM, not the heap the JVM allocates to the program it's running. – Scour