moving objects off heap

Question

Did anyone try to move java objects off heap? I tried using serializing, deserializing and storing byte arrays by using Unsafe libraries. But complex objects with multiple objects in it making this a tedious process. Any better solutions?

Urea

What are you trying to do? Maybe one of the various database systems would work? – Nitrite

What does "move off heap" mean? – Bruce

If you have complex mapping i would recommend to stick to Java serialization itself, You can store object state using JDBC for example using hibernate, but that requires complex mappings to be done which will further complicate your situation. – Athome

have you looked into apache directmemory? directmemory.apache.org Or mapdb.org – Immaterial

I have looked at Apache Direct memory. I am looking more into access disk space from Java program, something like C pointers and do "array of complex objects" operations – Urea

Answer 1

The unsafe field is your instance of Unsafe. If you don't know how to get it, simply use:

Unsafe unsafe = getUnsafe();
Unsafe getUnsafe() {
    try {
        Field f = Unsafe.class.getDeclaredField("theUnsafe");
        f.setAccessible(true);
        return (Unsafe) f.get(null);
    } catch (Exception x) {
        x.printStackTrace();
    }

    return null;
}

Keep in mind this should work for HotSpot JVM since the field name differs across different implementations.

You can move objects off the heap by:

1) Allocate the proper bytes to the memory address. You can read the int at the offset 4L, convert to unsigned long, and add 12L, and get the address at that location.

An example implementation:

public static long sizeOf(Object object) {
    return unsafe.getAddress( normalize( unsafe.getInt(object, 4L) ) + 12L );
}

public static long normalize(int value) {
   if(value >= 0) return value;
   return (~0L >>> 32) & value;
}

You can allocate the offheap space using:

Object object = // Your object
long size = sizeOf(object);
long address = unsafe.allocateMemory(size);

(Keep the address field handy)

2) Copy the object into the memory section specified by the address .

Example:

getUnsafe().copyMemory(
            object,
            0,
            null,
            address,
            size);

Done! Now your object is offheap.

To get your object back, simply execute following:

// Class wide field
Object anchor;

// Deserializarion
long offset = unsafe.objectFieldOffset(this.getClass().getDeclaredField("anchor"));

unsafe.putLong(this, offset, address);

Then, anchor should be your Object from offheap!

NOTE: Remember to release the resources of the allocated memory when you finish by doing:

unsafe.freeMemory(address);

(See, told you the address was important [as if it wasn't obvious already])

Source(s): http://highlyscalable.wordpress.com/2012/02/02/direct-memory-access-in-java/, a great guy named RawCode which I don't believe is here, but told me about how Unsafe works a while ago.

Karolyn

Thanks Dudex. I have seen this before and I quite dont understand the part " return unsafe.getAddress( normalize( unsafe.getInt(object, 4L) ) + 12L );" – Urea

That reads the number representing the size of the object in it's header hg.openjdk.java.net/jdk7u/jdk7u60/hotspot/file/ba66650acf63/src/… – Karolyn

I am trying to allocate memory for a byte stream after serializing and then retrieve deserialize and read. but I could not allocate memory properly this way...... byte[] abc = ser.serialize(ex); // ex is an object having two int and one string size = abc.length; address = unsafe.allocateMemory(size); unsafe.putObject(abc, address, abc ); – Urea

I think it would be best to open a new question. – Karolyn

Answer 2

There are some libraries which make this process at least less a fuss, at most store and more objects more efficiently to/from off-heap memory. Chronicle Values and Slab come to my mind first, but I remember there are some more.

Psalmody