Android ClassLoader memory leak

Question

Motivation:

I am using some native libraries in my Android application and I want to unload them from the memory at some point in time. Libraries get unloaded when ClassLoader that loaded class that loaded native libraries gets garbage collected. Inspiration: native unloading.

Problem:

• ClassLoader is not garbage collected if it is used to load some class (causes a possible memory leak).
• Native libraries can be loaded only in one ClassLoader in the application. If there is still old ClassLoader hanging somewhere in the memory and a new ClassLoader tries to load same native libraries at some point in time, exception is thrown.

Question:

1. How to perform unloading of a native library in a clean way (unloading is my ultimate target, no matter if it is a poor programming technique or something like that).
2. Why the memory leak appears and how to avoid it?

In the code below I simplify the case by omitting a native library loading code, just Classloader memory leak is demonstrated.

I am testing this on the Android KitKat 4.4.2, API 19. Device: Motorola Moto G.

For the demonstration I have the following ClassLoader, derived from PathClassLoader used for loading Android applications.

package com.demo;
import android.util.Log;
import dalvik.system.PathClassLoader;

public class LibClassLoader extends PathClassLoader { 
   private static final String THIS_FILE="LibClassLoader";

   public LibClassLoader(String dexPath, String libraryPath, ClassLoader parent) {
        super(dexPath, libraryPath, parent);
    }

    @Override
    protected void finalize() throws Throwable {
        Log.v(THIS_FILE, "Finalizing classloader " + this);
        super.finalize();
    }
}

I have the EmptyClass to load with the LibClassLoader.

package com.demo;
public class EmptyClass {
}

And the memory leak is caused in the following code:

final Context ctxt = this.getApplicationContext();
PackageInfo pinfo = ctxt.getPackageManager().getPackageInfo(ctxt.getPackageName(), 0);

LibClassLoader cl2 = new LibClassLoader(
    pinfo.applicationInfo.publicSourceDir,
    pinfo.applicationInfo.nativeLibraryDir,
    ClassLoader.getSystemClassLoader()); // Important: parent cannot load EmptyClass.

if (memoryLeak){
    Class<?> eCls = cl2.loadClass(EmptyClass.class.getName());
    Log.v("Demo", "EmptyClass loaded: " + eCls);
    eCls=null;
}

cl2=null;

// Try to invoke GC
System.runFinalization();
System.gc();
Thread.sleep(250);
System.runFinalization();
System.gc();
Thread.sleep(500);
System.runFinalization();
System.gc();
Debug.dumpHprofData("/mnt/sdcard/hprof"); // Dump heap, hardcoded path...

The important thing to note is that parent of the cl2 is not ctxt.getClassLoader(), the classloader that loaded the demonstration code class. This is by design because we don't want cl2 to use it's parent to load the EmptyClass.

The thing is that if memoryLeak==false, then cl2 gets garbage collected. If memoryLeak==true, memory leak appears. This behavior is not consistent with observations on standard JVM (I used class loader from [1] to simulate the same behavior). On JVM in both cases cl2 gets garbage collected.

I also analyzed heap dump file with Eclipse MAT, cl2 was not garbage collected because class EmptyClass still holds reference on it (as classes hold references on their class loaders). This makes sense. But EmptyClass was not garbage collected from no reason, apparently. Path to GC root is just this EmptyClass. I haven't managed to persuade GC to finalize EmptyClass.

HeapDump file for memoryLeak==true can be found here, Eclipse Android project with a demonstration application for this memory leak here.

I tried also another variations of loading the EmptyClass in the LibClassLoader, namely Class.forName(...) or cl2.findClass(). With/without static initialization, result was always the same.

I checked a lot of online resources, there are no static caching fields involved, as far as I know. I checked source codes of the PathClassLoader and it's parent classes and I found nothing problematic.

I would be very thankful for insights and any help.

Disclaimer:

• I accept this is not the best way of doing things, if there is any better option how to unload a native library, I would be more than happy to use that option.
• I accept that in general I cannot rely on the GC to be invoked in some time window. Even calling System.gc() is only a hint to execute GC for the JVM/Dalvik. I am just wondering why there is a memory leak.

Edit 11/11/2015

To make it more clear as Erik Hellman wrote, I am speaking about loading NDK compiled C/C++ library, dynamically linked, with .so suffix.

Headstream

Just a suggestion, Have you tried to change this line this.getApplicationContext(); to className.getApplicatonContext()? – Janettajanette

Answer 1

First, lets sort out the terminology here.

Is it a native library with JNI bindings that you want to load? That is, a file with the suffix .so that is implemented in C/C++ using the Android NDK? That is usually what we refer to when we talk about native library. If this is the case, then the only way to solve this is to run the library in separate process. Easiest way to do this is to create an Android service where you add android:process=":myNativeLibProcess" for the entry in the manifest. This Service will then call System.loadLibrary() as usual and you bind to the Service from your main process using Context.bindService().

If it is a set of Java classes inside a JAR file, then we are looking at something else. For Android, you need to create compile your library code into a DEX file that is placed into a JAR file and loaded using a DexClassLoader, similar to what you've done in your code. When you want to unload the library, you need to release all the references to the instances you've created AND the class loader used for loading the library. This will allow you to load a new version of the library later on. The only problem with this is that you won't reclaim all the memory used by the unloaded library on devices with API level 19 and lower (i.e., Android versions using Dalvik VM) because class definitions are not garbage collected. For Lollipop and later, the new VM will also garbage collect class definitions, so for these devices this will work better.

Hope that helps.

Ophthalmology

Do you have any explanation of the observed behavior seen by the OP? – Dynamics

Hard to tell definitively, but as I wrote in my answer, the class definitions are not garbage collected (pre Lollipop) even when the class loader is released. This could be the cause, since the OP tested this on KitKat. Testing this on Lollipop should confirm if it is the class definitions that are causing the "leak". – Ophthalmology

Thanks for the answer. I edited my original question to make it clear I mean NDK compiled C/C++ library. Using a completely different process is a good solution, by killing the process memory would be released, but for now it is not usable for us as some mobile device management systems do not support applications with multiple processes. But thanks anyway! – Headstream

An MDM supporting Android shouldn't even be aware of a multi-process application. All you need to do is to add android:process=":subProcess" to the Service and you're done. – Ophthalmology

@ErikHellman how do you finally kill the android sub-process, and can you start multiple sub-processes? – Escarole

Answer 2

May be you can find answers here

I am not sure that is this you are looking for but it gives actual library deallocating method in JVM.

Mezereon