How do I create a parent-last / child-first ClassLoader in Java, or How to override an old Xerces version that was already loaded in the parent CL?

Question

I would like to create a parent-last / child-first class loader, e.g. a class loader that will look for classes in the child class loder first, and only then delegate to it's parent ClassLoader to search for classes.

Clarification:

I know now that to get complete ClassLoading seperation I need to use something like a URLClassLoader passing null as it's parent, thanks to this answer to my previous question

However the current question comes to help me resolve this issue:

1. My code + dependent jars are being loaded into an existing system, using a ClassLoader that sets that System's ClassLoader as it's parent (URLClassLoader)

2. That System uses some libraries of a version not compatible with the one I need (e.g. older version of Xerces, that doesn't allow me to run my code)

3. My code runs perfectly fine if runs stand alone, but it fails if runs from that ClassLoader

4. Howerver I do need access to many other classes within the parent ClassLoader

5. Therefore I want to allow me to Override, the parent classloader "jars" with my own: If a class I call is found in the child class loader (e.g. I provided a newer version of Xerces with my own jars, instead of the one users by the ClassLoader that loaded my code and jars.

Here is the System's code that loads my code + Jars (I can't change this one)

File addOnFolder = new File("/addOns"); 
URL url = addOnFolder.toURL();         
URL[] urls = new URL[]{url};
ClassLoader parent = getClass().getClassLoader();
cl = URLClassLoader.newInstance(urls, parent);

Here is "my" code (taken fully from the Flying Sauser "Hello World" code demo):

package flyingsaucerpdf;

import java.io.*;
import com.lowagie.text.DocumentException;
import org.xhtmlrenderer.pdf.ITextRenderer;

public class FirstDoc {

    public static void main(String[] args) 
            throws IOException, DocumentException {

        String f = new File("sample.xhtml").getAbsolutePath();
        System.out.println(f);
        //if(true) return;
        String inputFile = "sample.html";
        String url = new File(inputFile).toURI().toURL().toString();
        String outputFile = "firstdoc.pdf";
        OutputStream os = new FileOutputStream(outputFile);

        ITextRenderer renderer = new ITextRenderer();
        renderer.setDocument(url);
        renderer.layout();
        renderer.createPDF(os);

        os.close();
    }
}

This works standalone (running main) but fails with this error when loaded through the parent CL:

org.w3c.dom.DOMException: NAMESPACE_ERR: An attempt is made to create or change an object in a way which is incorrect with regard to namespaces.

probably because the parent system uses Xerces of an older version, and even though I provide the right Xerces jar in the /addOns folder, since it's classes were already loaded and used by the parent System, it doesn't allow my own code to use my own jar due to the direction of the delegation. I hope this makes my question clearer, and I'm sure it has been asked before. (Perhaps I don't ask the right question)

Schreiner

If I were you, I would rephrase the question a bit to avoid having it closed as a duplicate of stackoverflow.com/questions/5444246 – Truett

This is against the specifications and can lead to some complications if you don't pay attention to which classes are where, but there is nothing hindering you. What was your question, again? – Typewritten

@Truett - I know, I wrote it, but the other question was - "How to create a class loader that doesn't delegate to the parent", I got the answer, but realized that for what I need, I need to do a parent-last, e.g. one that does delegate to the parent, but only after searching the child, e.g. current. Just like WebLogic does, and any decent multiple application container. E.g. WebLogic uses Xerces version X but I like version Y, so WebLogic doing parent-last is again the specifications? Please provide a reference that confirms that, I'd like to ask IBM's tech support what they make of it. – Schreiner

Is stackoverflow only for jQuery and .NET questions? Or is this question so niche that it get's no interest? How come philosophical questions on SO such as "When good programmers go bad!" gets thousands of hits where this question, that seems to have no definite or complete answer anywhere gets so little traffic? (or I'm using the wrong keywords) it seems to be the first result on google for parent-first classloader! I'm starting to think I'm missing something big, or no one has this problem because I'm doing it all wrong in the first place... troubling... – Schreiner

Feeling a bit impatient? Some of us do things on Saturday evening other than wait around for you to clarify a question. – Monosaccharide

Better than answering questions on StackOverflow? I can't imagine :) So double my gratitude for clarifying your answer on saturday night nevertheless! – Schreiner

@Eran Medan I know you wrote it, that's why I suggested clarifying what you are after. There are ways to do things like this but all they ways I've done it has had drawbacks I don't like recommending. I agree you should look at something like Tomcat which needs to do this in order to reload jsp and servlets but you IMHO should ask yourself first if you're heading down the right path or if you can avoid this. – Truett

The question does not really make sense. This does not look like a classlaoder problem but just like a classpath problem. How can you laod "the wrong Xerxes classes?" Just remove the wrong ones from your classpath. THIS is why there is no "traffic" as you say so ;D – Primeval

Answer 1

Today is your lucky day, as I had to solve this exact problem. I warn you though, the innards of class loading are a scary place. Doing this makes me think that the designers of Java never imagined that you might want to have a parent-last classloader.

To use just supply a list of URLs containing classes or jars to be available in the child classloader.

/**
 * A parent-last classloader that will try the child classloader first and then the parent.
 * This takes a fair bit of doing because java really prefers parent-first.
 * 
 * For those not familiar with class loading trickery, be wary
 */
private static class ParentLastURLClassLoader extends ClassLoader 
{
    private ChildURLClassLoader childClassLoader;

    /**
     * This class allows me to call findClass on a classloader
     */
    private static class FindClassClassLoader extends ClassLoader
    {
        public FindClassClassLoader(ClassLoader parent)
        {
            super(parent);
        }

        @Override
        public Class<?> findClass(String name) throws ClassNotFoundException
        {
            return super.findClass(name);
        }
    }

    /**
     * This class delegates (child then parent) for the findClass method for a URLClassLoader.
     * We need this because findClass is protected in URLClassLoader
     */
    private static class ChildURLClassLoader extends URLClassLoader
    {
        private FindClassClassLoader realParent;

        public ChildURLClassLoader( URL[] urls, FindClassClassLoader realParent )
        {
            super(urls, null);

            this.realParent = realParent;
        }

        @Override
        public Class<?> findClass(String name) throws ClassNotFoundException
        {
            try
            {
                // first try to use the URLClassLoader findClass
                return super.findClass(name);
            }
            catch( ClassNotFoundException e )
            {
                // if that fails, we ask our real parent classloader to load the class (we give up)
                return realParent.loadClass(name);
            }
        }
    }

    public ParentLastURLClassLoader(List<URL> classpath)
    {
        super(Thread.currentThread().getContextClassLoader());

        URL[] urls = classpath.toArray(new URL[classpath.size()]);

        childClassLoader = new ChildURLClassLoader( urls, new FindClassClassLoader(this.getParent()) );
    }

    @Override
    protected synchronized Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException
    {
        try
        {
            // first we try to find a class inside the child classloader
            return childClassLoader.findClass(name);
        }
        catch( ClassNotFoundException e )
        {
            // didn't find it, try the parent
            return super.loadClass(name, resolve);
        }
    }
}

EDIT: Sergio and ɹoƃı have pointed out that if you call .loadClass with the same classname, you will get a LinkageError. While this is true, the normal use-case for this classloader is to set it as the thread's classloader Thread.currentThread().setContextClassLoader() or via Class.forName(), and that works as-is.

However, if .loadClass() was needed directly, this code could be added in the ChildURLClassLoader findClass method at the top.

                Class<?> loaded = super.findLoadedClass(name);
                if( loaded != null )
                    return loaded;

Biyearly

Thank you! by the way, doesn't the fact you had to write it by yourself make you worried a little? I mean, if it is such a common problem, and such a complex solution, someone (e.g. like you) would have added it to something like org.apache.commons.lang3.classloading or something, isn't it so? My concern is, that there is another way around this or a better modular way to do things (e.g. OSGI). By the way, why don't you put this as an open source library in sourceforge or something :) both you (since you wrote it) and I (since I'm asked) googled "parent last classloader" and didn't find one... – Schreiner

And thanks again for sharing your code, I'll let you know if it works for me! (is it only me or Stackoverflow's syntax coloring 3rd party makes it very hard to copy and paste code, is it only me that get's it all under a single line in eclipse / notepad and I have to paste in MS Word to get it right? I'm sure I'll find something on this in meta...) – Schreiner

You're welcome. It's a good question why this isn't in an open source library. Perhaps it is and I (and you) couldn't find it. Someday when I get a free moment I might go into Tomcat's source and see how they do it, probably in a not too dissimilar way. – Biyearly

Ok, told you it will be apache eventually, it seems that AntClassLoader has a support for parent-last: svn.apache.org/repos/asf/ant/core/trunk/src/main/org/apache/… – Schreiner

@Biyearly - I like the elegance and quality of your code, but why do you override findClass in FindClassClassLoader, isn't an override that calls the superclass implementation of the same method actually is the same as not overriding it? or am I missing something? – Schreiner

Did you consider this scenario? Search child CL first, but then search System CL before you search the parent to avoid overriding J2SE libraries by your parent that you don't want overriden – Schreiner

Thanks for the compliment. ClassLoader#findClass is protected and I needed a way to call it on an existing class loader. There may well be better ways to do this, this is just what I came up with after a lot of trial and error. – Biyearly

There is problem in not caching loaded class in loadClass, as calling findClass twice on same class name will result in LinkageError: attempted duplicate class definition.... – Straggle

Agreed with @ɹoƃı ... I've experienced the same problem .... the findClass method should first check if the class we're trying to find was already loaded via this.findLoadedClass(name) and only if this call fails it should call super.findClass(name). See ibm.com/developerworks/java/library/j-dclp3 - Cache section. – Thrasher

@Sergio you are correct in that if you call loadClass twice with the same class you will get a LinkageError. However if you use the classloader normally (like setContextClassLoader/Class.forName), this works perfectly. I'll add a note in the answer. – Biyearly

I had issues with loading JDBC drivers. It looks like that a getParent() on ChildURLClassLoader is used. When I changed super(urls, null) into super(urls, realParent) it worked. Unfortunate one can not override getParent(). The code example below from Yoni does not have this issue. – Rancor

What is the purpose of the outside class, ParentLastURLClassLoader? Wouldn't it be possible to use the ChildURLClassLoader directly (with minor changes)? – Karakorum

It's also worth noting that you might want to implement URL findResource(String name) as well (delegating to childClassLoader as well) otherwise any calls from classes loaded by your new CL to methods such as getResourceAsStream will fail and return null. – Torrential

Answer 2

The following code is what I use. It has the advantage over the other answer that it doesn't break the parent chain (you can follow getClassLoader().getParent()).

It also has an advantage over tomcat's WebappClassLoader by not reinventing the wheel and not depending on other objects. It re-uses code from URLClassLoader as much as possible.

(it doesn't yet honor the system class loader, but when I get that fixed I'll update the answer)

It honors the system class loader (for java.* classes, endorsed dir, etc.). It also works when security is turned on and the classloader doesn't have access to its parent (yes, this situation is weird, but possible).

public class ChildFirstURLClassLoader extends URLClassLoader {

    private ClassLoader system;

    public ChildFirstURLClassLoader(URL[] classpath, ClassLoader parent) {
        super(classpath, parent);
        system = getSystemClassLoader();
    }

    @Override
    protected synchronized Class<?> loadClass(String name, boolean resolve)
            throws ClassNotFoundException {
        // First, check if the class has already been loaded
        Class<?> c = findLoadedClass(name);
        if (c == null) {
            if (system != null) {
                try {
                    // checking system: jvm classes, endorsed, cmd classpath, etc.
                    c = system.loadClass(name);
                }
                catch (ClassNotFoundException ignored) {
                }
            }
            if (c == null) {
                try {
                    // checking local
                    c = findClass(name);
                } catch (ClassNotFoundException e) {
                    // checking parent
                    // This call to loadClass may eventually call findClass again, in case the parent doesn't find anything.
                    c = super.loadClass(name, resolve);
                }
            }
        }
        if (resolve) {
            resolveClass(c);
        }
        return c;
    }

    @Override
    public URL getResource(String name) {
        URL url = null;
        if (system != null) {
            url = system.getResource(name); 
        }
        if (url == null) {
            url = findResource(name);
            if (url == null) {
                // This call to getResource may eventually call findResource again, in case the parent doesn't find anything.
                url = super.getResource(name);
            }
        }
        return url;
    }

    @Override
    public Enumeration<URL> getResources(String name) throws IOException {
        /**
        * Similar to super, but local resources are enumerated before parent resources
        */
        Enumeration<URL> systemUrls = null;
        if (system != null) {
            systemUrls = system.getResources(name);
        }
        Enumeration<URL> localUrls = findResources(name);
        Enumeration<URL> parentUrls = null;
        if (getParent() != null) {
            parentUrls = getParent().getResources(name);
        }
        final List<URL> urls = new ArrayList<URL>();
        if (systemUrls != null) {
            while(systemUrls.hasMoreElements()) {
                urls.add(systemUrls.nextElement());
            }
        }
        if (localUrls != null) {
            while (localUrls.hasMoreElements()) {
                urls.add(localUrls.nextElement());
            }
        }
        if (parentUrls != null) {
            while (parentUrls.hasMoreElements()) {
                urls.add(parentUrls.nextElement());
            }
        }
        return new Enumeration<URL>() {
            Iterator<URL> iter = urls.iterator();

            public boolean hasMoreElements() {
                return iter.hasNext(); 
            }
            public URL nextElement() {
                return iter.next();
            }
        };
    }

    @Override
    public InputStream getResourceAsStream(String name) {
        URL url = getResource(name);
        try {
            return url != null ? url.openStream() : null;
        } catch (IOException e) {
        }
        return null;
    }

}

Isolated

What is the license for the source code of ChildFirstURLClassLoader in this solution? – Dicephalous

Feel free to use it, it's public domain as far as I am concerned – Isolated

Could you elaborate on why you had to override getResourceAsStream(String)? The super impl will still use the getResource(String) impl of this class, so why do you need to implement it yourself, and lose functionality that the super will give you for free? – Diopside

Works well except for the case where the parent classloader IS the system classloader (i.e., the child override functionality is short circuited). – Bandmaster

Answer 3

By reading the source code of either Jetty or Tomcat, both of which provide parent-last class loaders to implement webapp semantics.

https://github.com/apache/tomcat/blob/7.0.93/java/org/apache/catalina/loader/WebappClassLoaderBase.java

Which is to say, by overriding the findClass method in your ClassLoader class. But why reinvent the wheel when you can steal it?

Reading your various updates, I see that you ran into some classic problems with the XML SPI system.

The general problem is this: if you create a completely isolated class loader, then it's hard to use the objects it returns. If you allow sharing, you can have problems when the parent contains the wrong versions of things.

It is to deal with all this lunacy that OSGi was invented, but that's a big pill to swallow.

Even in webapps, the class loaders exempt some packages from the 'local-first' processing on the assumption that the container and the webapp have to agree on the API between them.

Monosaccharide

I agree, but how come I can't find either a code sample, or a class name from an existing library jar, am I asking the wrong questions? – Schreiner

I wish I could up vote twice, I wish there were more trafic to this question... (even though it's saturday night, or actually midnight) – Schreiner

Answer 4

(see at the bottom for an update on a solution I found)

It seems that AntClassLoader has a support for parent first/last, (didn't test it yet)

http://svn.apache.org/repos/asf/ant/core/trunk/src/main/org/apache/tools/ant/AntClassLoader.java

Here is a snippet

/**
 * Creates a classloader for the given project using the classpath given.
 *
 * @param parent The parent classloader to which unsatisfied loading
 *               attempts are delegated. May be <code>null</code>,
 *               in which case the classloader which loaded this
 *               class is used as the parent.
 * @param project The project to which this classloader is to belong.
 *                Must not be <code>null</code>.
 * @param classpath the classpath to use to load the classes.
 *                  May be <code>null</code>, in which case no path
 *                  elements are set up to start with.
 * @param parentFirst If <code>true</code>, indicates that the parent
 *                    classloader should be consulted  before trying to
 *                    load the a class through this loader.
 */
public AntClassLoader(
    ClassLoader parent, Project project, Path classpath, boolean parentFirst) {
    this(project, classpath);
    if (parent != null) {
        setParent(parent);
    }
    setParentFirst(parentFirst);
    addJavaLibraries();
}

Update:

Found this as well, when as a last resort I started guessing class names in google (this is what ChildFirstURLClassLoader produced) - but it seems to be incorrect

Update 2:

The 1st Option (AntClassLoader) is very coupled to Ant (requires a Project context and not easy to pass a URL[] to it

The 2nd Option (from an OSGI project in google code) wasn't quite what I needed as it searched parent classloader before the system classloader (Ant class loader does it correctly by the way). The problem as I see it, think that your parent classloader includes a jar (that it shouldn't have) of a functionality that wasn't on JDK 1.4 but was added in 1.5, this has no harm as the parent last class loader (regular delegation model, e.g. URLClassLoader) will always load first the JDK's classes, but here the child first naive implementation seems to unveil the old, redundant jar in the parent class loader, shadowing the JDK / JRE own implementation.

I have yet to find a certified, fully tested, mature Parent Last / Child First correct implementation that is not coupled to a specific solution (Ant, Catalina/Tomcat)

Update 3 - I found it! I WAS looking in the wrong place,

All I did was add META-INF/services/javax.xml.transform.TransformerFactory and restored the JDK's com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl instead of the old Xalan's org.apache.xalan.processor.TransformerFactoryImpl

The only reason I don't "accept my own answer" yet, is that I don't know if the META-INF/services approach has the same classloader delegation as regular classes (e.g. is it parent-first / child-last or parent-last / child-first?)

Schreiner

Answer 5

URLClassLoader had this constructor public URLClassLoader(URL[], ClassLoader) that allows you to override the parent classloader of an URLClassLoader. You can just load your classloader through an URLClassLoader with an overridden parent classloader.

Areaway

Answer 6

You can override findClass() and loadClass() to implement a child first class loader:

/**
 * Always throws {@link ClassNotFoundException}. Is called if parent class loader
 * did not find class.
 */
@Override
protected final Class findClass(String name)
        throws ClassNotFoundException
{
    throw new ClassNotFoundException();
}

@Override
protected Class loadClass(String name, boolean resolve)
        throws ClassNotFoundException
{
    synchronized (getClassLoadingLock(name)){
        /*
         * Check if we have already loaded this class.
         */
        Class c = findLoadedClass(name);

        if (c == null){
            try {
                /*
                 * We haven't previously loaded this class, try load it now
                 * from SUPER.findClass()
                 */
                c = super.findClass(name);
            }catch (ClassNotFoundException ignore){
                /*
                 * Child did not find class, try parent.
                 */
                return super.loadClass(name, resolve);
            }
        }

        if (resolve){
            resolveClass(c);
        }

        return c;
    }
}

Nagle

Answer 7

A little bit late to the party but none of the answers helped my problem as I was getting linkage errors. Here is what worked for me:

import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.ArrayList;
import java.util.Enumeration;
import java.util.Iterator;
import java.util.List;


public class ParentLastURLClassLoader extends URLClassLoader {

    /**
     * Create a child-first/parent-last URLClassLoader
     * @param urls
     *  the URLS to be included in the classpath of the new classloader
     * @param parent
     *  parent classloader
     */
    public ParentLastURLClassLoader(URL[] urls, ClassLoader parent) {
        super(urls, parent);
    }

    @Override
    /**
     * {@inheritDoc}
     * 
     * For the parent-last classloader, first, if the class is already loaded, just recovers
     * it, and if it has to laod it, it tries first to load the class from the JARs,
     * and if it is not possible, it falls back to the parent classloader
     */
    protected Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
        // First, check if the class is already loaded
        Class<?> loadedClass = findLoadedClass(name);
        if (loadedClass != null) {
            return loadedClass;
        }

        try {
            // Try to load the class from the URLs of this classloader
            Class<?> localClass = findClass(name);
            if (resolve) {
                resolveClass(localClass);
            }
            return localClass;
        } catch (ClassNotFoundException e) {
            // Class not found in this classloader, delegate to parent classloader
            return super.loadClass(name, resolve);
        }
    }
}

UPDATE The code from @reda-alaoui works perfectly and it is better than mine: https://gist.github.com/reda-alaoui/a3030964293268eca48ddc66d8a07d74

Truth