Library shutdown routine that works well in a "normal" Java application and in a web application
Asked Answered
F

1

12

I maintain a JDBC driver that also has an embedded database server mode provided through a native library, which is accessed through JNA. The shutdown process, done as part of unloading the native library itself, runs into problems on Windows due to the order of unloading its dependencies. To avoid access violations or other issues, I need to explicitly shut down the embedded engine before unloading this library.

Due to the nature of its use, it is difficult to determine an appropriate moment to call for a shutdown. The only correct way to do this for a normal Java application is by registering a shutdown hook using

Runtime.getRuntime().addShutdownHook with a subclass of Thread that implements the shutdown logic.

This works fine for a normal Java application, but for web applications that include my library as part of the application (in the WEB-INF/lib of the WAR), this will cause a memory leak on undeploy as the shutdown hook will maintain a strong reference to my shutdown implementation and to the classloader of the web application.

What would be a suitable and appropriate way to address this? Options I'm looking into right now are:

  • Using java.sql.DriverAction.deregister() to do the cleanup.

    Not suitable as a driver will not be deregistered on a normal application exit.

  • Using java.sql.DriverAction.deregister() to remove the shutdown hook and execute the shutdown logic itself.

    Use of DriverAction is slightly problematic given the driver still supports Java 7, and this class was introduced in JDBC 4.2 (Java 8). This is technically not always the correct use of action (a JDBC driver can also be deregistered while existing connections remain valid and in use), and it is possible that the driver is used (through a javax.sql.DataSource) while the JDBC java.sql.Driver implementation is not registered.

  • Including a javax.servlet.ServletContextListener implementation annotated with @WebListener with the driver that will remove the shutdown hook and execute the shutdown logic itself.

    This option has complications if the driver is deployed to the server as a whole instead of to a specific web application (although those complications can be solved).

Is there a shutdown mechanism in Java I have overlooked that could be suitable for my needs?

Fogle answered 4/8, 2018 at 11:29 Comment(5)
Could you please explain what you mean by the driver being deployed to the server as a whole? I could not find anything about it, and I'm not aware of such functionality (at least not in Tomcat). I currently use the @WebListener-based solution in Tomcat to deregister a few java.sql.Drivers (using DriverManager.deregisterDriver) in a few containers (I always deregister exactly the driver that got registered by given container by storing Class<? extends java.sql.Driver>), and I'm wondering whether I haven't overlooked something.Danialah
@TomaszLinkowski You can deploy a driver so the driver is globally available (possibly used as a server-wide data source). For example in Tomcat if you put it in <catalina-home>/lib and define a data source in server.xml. Similar features exist in other application servers. Manually registering/deregistering the driver won't work for those situations (nor will this work for non-web applications), and I don't think it will solve my problem if the driver is globally available even if registered per WAR (java.sql.Driver implementations are (or should be) pretty light-weight).Fogle
@TomaszLinkowski re: light-weight: a driver itself doesn't actually 'hold' much, so the rest of the implementation could be shared, so deregistering my native library in that situations will probably be a bad idea (or require some additional reflection magic). Given the lack of responses so far, it looks like there is no mechanism that I can apply in all situations, so I will have to find some hybrid approach.Fogle
Thanks for the explanation :) Well, I understand you need to do more than I do (I just deregister the drivers so that Tomcat doesn't complain upon redeploys). As far as I understand, you want to shutdown the embedded engine only if no drivers are registered anymore, right? So e.g. if there's no global driver, and you have two containers, and each registered its driver, then if one container is destroyed, you do not want to shutdown the embedded engine until the second one is destroyed, right? And if there is a global engine, you want to shutdown only on app server shutdown, is that so?Danialah
@TomaszLinkowski That sounds about right, except for the 'no global driver' situation, the embedded engine will need to be shutdown on container destruction, as the embedded engine in that case will be loaded per container.Fogle
D
2

I tried to figure this out as this seems such an interesting case. I'm posting my findings here, although I feel I might still have misunderstood something, or made some too far-fetched simplifications. Actually, it's also possible that I totally misunderstood your case, and this answer is all useless (if so, I apologize).

What I assembled here is based on two notions:

  • application server global state (I use System.props, but it might not be the best choice - perhaps some temporary files would do better)
  • container-specific global state (which means all classes loaded by the container-specific ClassLoader)

I propose an EmbeddedEngineHandler.loadEmbeddedEngineIfNeeded method that would be called:

  • during your driver registration
  • in your javax.sql.DataSource implementation static initializer (if this whole DataSource-related thing works that way - I know little about it)

If I got it right, you won't need to call Runtime.removeShutdownHook at all.

The main thing that I'm uncertain about here is this - if the driver is deployed globally, would it be registered before any servlet is initialized? If not, then I got it wrong, and this won't work. But maybe inspecting the ClassLoader of EmbeddedEngineHandler could help then?


This is the EmbeddedEngineHandler:

final class EmbeddedEngineHandler {

    private static final String PREFIX = ""; // some ID for your library here
    private static final String IS_SERVLET_CONTEXT = PREFIX + "-is-servlet-context";
    private static final String GLOBAL_ENGINE_LOADED = PREFIX + "-global-engine-loaded";

    private static final String TRUE = "true";

    private static volatile boolean localEngineLoaded = false;

    // LOADING
    static void loadEmbeddedEngineIfNeeded() {
        if (isServletContext()) {
            // handles only engine per container case
            loadEmbeddedEngineInLocalContextIfNeeded();
        } else {
            // handles both normal Java application & global driver cases
            loadEmbeddedEngineInGlobalContextIfNeeded();
        }

    }

    private static void loadEmbeddedEngineInLocalContextIfNeeded() {
        if (!isGlobalEngineLoaded() && !isLocalEngineLoaded()) { // will not load if we have a global driver
            loadEmbeddedEngine();
            markLocalEngineAsLoaded();
        }
    }

    private static void loadEmbeddedEngineInGlobalContextIfNeeded() {
        if (!isGlobalEngineLoaded()) {
            loadEmbeddedEngine();
            markGlobalEngineAsLoaded();
            Runtime.getRuntime().addShutdownHook(new Thread(EmbeddedEngineHandler::unloadEmbeddedEngine));
        }
    }

    private static void loadEmbeddedEngine() {
    }

    static void unloadEmbeddedEngine() {
    }

    // SERVLET CONTEXT (state shared between containers)
    private static boolean isServletContext() {
        return TRUE.equals(System.getProperty(IS_SERVLET_CONTEXT));
    }

    static void markAsServletContext() {
        System.setProperty(IS_SERVLET_CONTEXT, TRUE);
    }

    // GLOBAL ENGINE (state shared between containers)
    private static boolean isGlobalEngineLoaded() {
        return TRUE.equals(System.getProperty(GLOBAL_ENGINE_LOADED));
    }

    private static void markGlobalEngineAsLoaded() {
        System.setProperty(GLOBAL_ENGINE_LOADED, TRUE);
    }

    // LOCAL ENGINE (container-specific state)
    static boolean isLocalEngineLoaded() {
        return localEngineLoaded;
    }

    private static void markLocalEngineAsLoaded() {
        localEngineLoaded = true;
    }
}

and this is the ServletContextListener:

@WebListener
final class YourServletContextListener implements ServletContextListener {

    @Override
    public void contextInitialized(ServletContextEvent sce) {
        EmbeddedEngineHandler.markAsServletContext();
    }

    @Override
    public void contextDestroyed(ServletContextEvent sce) {
        if (EmbeddedEngineHandler.isLocalEngineLoaded()) {
            EmbeddedEngineHandler.unloadEmbeddedEngine();
        }
    }
}
Danialah answered 15/8, 2018 at 20:40 Comment(6)
This looks like a promising solution, thanks. I can see some possible pain points to implement this, but I think this may work.Fogle
I took a while to get to this, but I have finally implemented it: github.com/FirebirdSQL/jaybird/commit/… Thanks again for your help.Fogle
@MarkRotteveel I'm glad you managed to solve the problem. I see it's a bit different to what I proposed (e.g. you use removeShutdownHook, and you compare ClassLoaders to determine if the driver was loaded in a servlet context or not), but I trust you had good reasons for such an approach :)Danialah
Yes, I modified your approach to solve issues when the driver is on the main classpath, but only first accessed from within the servlet context. With your approach that would lead to issues if the driver was used in multiple contexts, and one was stopped or redeployed. My current solution works in Tomcat, although I still have to test it in other application servers (checking the classloaders might be too naive).Fogle
I see, that's what I was afraid might be the case when I wrote: "if the driver is deployed globally, would it be registered before any servlet is initialized?". Nice that you managed to work around it!Danialah
Yes, it took some trial and error to get it to work, but your answer gave me a good starting point. And in just in case I messed things up (or didn't consider certain cases), I also defined a system property that disables this cleanup/shutdown feature entirely.Fogle

© 2022 - 2024 — McMap. All rights reserved.