Using `META-INF/services` for internal plumbing of driver

Question

I develop the Jaybird JDBC driver, and today I came across an issue (JDBC-325, How to configure Jaybird with hibernate) that is related to how Jaybird loads some of its components and how - in this case - NetBeans restricts classloading.

The issue is related to the way Jaybird loads parts of itself using entries in META-INF/services and the that a classloader used by NetBeans for a Hibernate wizard explicitly ignores those files (see details below).

I can workaround this issue by (also) attempting to load a hardcoded list of plugins that are part of the Jaybird implementation, or by moving the definition to a different location.

However I was wondering if it is weird (or wrong) to use META-INF/services for internal purposes like Jaybird does?

I also don't understand why would NetBeans would exclude loading of META-INF/services? The comment by Drew seems to indicate NetBeans used it to solve errors when loading a driver (see this issue), although I would think that would be better solved by the user including all dependencies of a driver.

Details of the problem

Jaybird uses plugins for the supported protocols, for example the Type 4 protocol, a custom Type 4 Open Office protocol, the Type 2 embedded (native) protocol and the Type 2 native client protocol. I also believe a third party once used it to provide a driver that translated Oracle specific syntax to Firebird syntax.

All these plugins are listed in META-INF/services/org.firebirdsql.gds.impl.GDSFactoryPlugin and are loaded in a way that is similar to java.util.ServiceLoader (current 2.2.x drivers still support Java 5 so we don't actually use ServiceLoader). For the upcoming version I was also planning to use this for the supported connection encodings and (wire) protocol definitions. This would allow for 'custom' encoding definitions (eg extend the supported encodings, or use an alternate encoding) or different protocol implementation (eg for troubleshooting, custom logging etc).

Now the actual problem is that the Netbeans wizard Hibernate Mapping Files and POJOs from Database uses a custom classloader (org.netbeans.modules.hibernate.util.CustomClassLoader), and this classloader ignores files in META-INF/services. Note that it is only this Wizard that has issues, Netbeans itself can use the driver without problems.

Code ignoring META-INF/services:

@Override
public URL findResource(String name) {
    return name.startsWith("META-INF/services") ? null : super.findResource(name); //NOI18N
}

@Override
public Enumeration<URL> findResources(String name) throws IOException {
    if (name.startsWith("META-INF/services")) { //NOI18N
        return Collections.enumeration(Collections.<URL>emptyList());
    } else {
        return super.findResources(name);
    }
}

This results in no plugins being discovered and the driver has no protocols, which leads to a NullPointerException inside Netbeans because no connection is created.

Undergo

It does seem weird to me. Those lines appeared in hg.netbeans.org/main-silver?cmd=changeset;node=c8413dd446c2 as a response to netbeans.org/bugzilla/show_bug.cgi?id=194915. Looks like to fix/avoid one bug the change leads to your problem. – Exeat

Can you just use a different filename? META-INF/jaybird for example? I understand that it is a NetBeans bug, but waiting for them to fix it will be like Waiting for Godot. – Calebcaledonia

@Calebcaledonia Yes I could do that, but META-INF/services is the standard way to do this. Also I want to switch to the standard java.util.ServiceLoader to load these classes instead of the own implementation Jaybird had for compatibility with Java 5 and older, and ServiceLoader only loads things from META-INF/services. For now I have simply added a fallback mechanism which loads the default list if none were loaded using the services definition. – Undergo

Answer 1

I think that Netbeans team fix to the bug was wrong. Ignoring files in a specific directory for no particular reason is terrible. Mainly in such an important directory like META-INF/services. It's not a security issue or anything like that. They are only protecting them for other people's bad written code. They should use some other way to do that. I can only imagine how long someone like you took to find the cause of this issue!

The Service Provider API is public for one reason: everyone should be using it! It's a great way to make your code less coupled and it works great! I use it everytime I can and suggest everyone to use it.

And the Java API is clearly supporting adding JDBC drivers using the Service Provider mechanism:

The DriverManager methods getConnection and getDrivers have been enhanced to support the Java Standard Edition Service Provider mechanism. JDBC 4.0 Drivers must include the file META-INF/services/java.sql.Driver.

That also states that drivers created after JDBC 4.0 (2007) are expected to provide an entrance through that mechanism.

It does not state though that you should/can not provide a fallback. Other drivers must be doing that otherwise they would be having the same issue. But they probably do that for other reasons (supporting older versions of the JDBC API).

So you're doing the right thing and if supporting that specific use case is so important for you, then you will need to maintain that code as fallback. Otherwise, remove the code and add some documentation so that people can work around it.

Alford

Thanks for your answer, and the confirmation that we aren't doing anything weird. The problem I describe is not with the java.sql.Driver loading mandated by the JDBC specification, although it would be affected in the same way if Netbeans didn't require explicitly specifying the driver class! The actual problem is with enumerating (and then loading) plugins/providers internal to the driver. – Undergo

@MarkRotteveel I understand that your problem is not with java.sql.Driver. That's why I edited my answer to make it less specific. I used the java.sql.Driver just as an example why Netbeans' team choice was not ideal. If I didn't answer your question, can you please tell me what am I missing so that I can add to it? – Alford

I had only seen the final revision, it answers my question and I have now accepted it. The second part of your answer doesn't entirely fit, but having thought it over (and seen the earlier revisions) I get your point :) Thanks again. – Undergo