When and how is a java classloader marked for garbage collection?

Question

We are creating multiple child classloaders to load in multiple subapplications into a Java application "container", prototyping hot deployment. When the classpath of a particular classloader has changed (i.e. jars have been added, deleted, updated), the old classloader is thrown away (unreferenced) and a new classloader is created for the new classpath of jars.

After updating the classpath, triggering the hot deployment, we took a heap dump. The heap dump (using Memory Analyzer) indicates that the old classloaders were not being garbage collected. Certain classes in the parent classloader were caching the old classloaders. The following things were invoked to clear these caches:

java.lang.ResourceBundle.clearCache(classLoader);
org.apache.commons.logging.LogFactory.release(classLoader);
java.beans.Introspector.flushCaches();

Even after clearing the above caches, the old classloader were still not being garbage collected. The remaining references to the classloader included the following:

• the classes loaded by the classloader
• java.lang.Package's created by the classloader itself
• java.lang.ProtectionDomain created by the classloader itself

All the above are circular references within the classloader, which should trigger a garbage collection. I'm not sure why it is not. Does anybody know why the old classloaders are still not being garbage collected even with the circular references?

Alegre

Which JVM do you use (exact version)? Do you use any JVM options, that might affect classloading? Do you use any stuff from Sun's own implementations? Does the application manipulate byte code? ... What's the environment, that might affect class loading? – Vilayet

Not related to your main question, but did you consider something like OSGi instead of doing your own framework that supports hot deployment? – Baptistry

@bfoo In our tests, we are using Java 6. No JVM options. We are not using Sun's impl of anything in the simplest case. No byte code manipulation. – Alegre

@stevendick Yes we considered OSGi, but OSGI seemed to be overkill for our use case. The main hurdle to overcome was to avoid conflicts of third party libraries used by different sub-applications. A subapplication would use a completely equivalent libraries but in different classloaderes to avoid any potential conflicts and complexities. That does sound like it would use too much memory... Have you used OSGi? And if so, what are some of hurdles you faced? – Alegre

To be clear, with usage of Sun implementations I mean to e.g. use socket communication with serialization, like RMI / HTTP/SOAP calls et cetera. Those implementations might use caching methodologies e.g. for serialization issues. This way, your classes (to be unloaded) are referenced by ClassLoaders above and/or aside the ClassLoader you like to have purged. Those implementations might use strong references to the classes in order to ensure serialization would not raise system/VM IO issues. – Vilayet

I standard GC cycle wont look at the permanent generation. Can you go as far as getting an OOME. – Prakash

Well, first things first, how can you be sure that your stuff isn't being garbage collected? From your description, it just looks like you unreferenced a Classloader. That does not mean that poof the heap gets cleaned of it's existence. In 2010, you were probably using the Parallel GC, which means the GC event doesn't trigger until one of the spaces (young or old) gets filled. Meaning your Classloader will still be on the heap well after you dereference it, until a space gets filled. Newer GCs in 2022 are a bit more proactive, but it still remains that just dereferencing doesnt deallocate. – Timoshenko

Answer 1

I always heard that Classloader unloading was problematic. They are theoretically garbage collected when there is not reference to the object instances and class unloading is not necessary, but in practice it seems like to be more problematic. Subtle references may leak and prevent the Classloader from being reclaimed. In application servers, after numerous redeploy cycle, I sometimes got a OutOfMemoryError: PermGen space.

All that to say that I guess there is a nasty reference somewhere that prevent it from being collected -- maybe the memory analyzer didn't followed the link correctly. It seems like all this can happen, as described in these articles:

• Classloader leaks: the dreaded PermGen space exception
• How to fix the dreaded PermGen space exception

Also, I don't know exactly what you are doing, but if you can wait for JDK 7, you could have a look at AnonymousClassLoader. They will be introduced to better support dynamic language, as explained in this post:

• A first taste of InvokeDynamic

I hope it will help you.

Amok

One sub-application is very simple. The only third party library it uses is log4j. Nothing else. The rest is the basic JDK API. It also uses the ResourceBundle class which keeps cache. After calling clearCache(), those references did disappear, but it still does not get garbage collected. But Thanks for the answer :) – Alegre

oh and by the way, great links! – Alegre

I can only suggest to try with the simplest sub-app ever (say with one class) and then add more and more classes and dependencies to see at which point it stops the GC of the classloader. But I know it can be extremely time-consuming. BTW, Frank Kievet wrote also interesting posts on XA transactions, if you're interested. – Amok

you are correct. A nasty reference is preventing it from being collected. However, the Eclipse Memory Analyzer was after all displaying all the references correctly. The actual architecture of the system is bit more complex than I described above, written for the brevity and simplicity of the question. I will provide further details in an answer. – Alegre

Glad to hear that there is not bug in the GC and it's indeed a leak :) – Amok

More useful reading on this topic: management-platform.blogspot.com/2009/01/… – Supplement

@ewernli, In your first paragraph, you stated: "I always heard that Classloader unloading was problematic". Do you mean that JVM has problems doing it, or do you mean that programmers has problem implementing it correctly, but when its done correctly JVM has no problems doing it? – Tidal