Call a non-reentrant native shared library from multiple Java threads

Question

I have some Java code that is calling some native code, originally written in Fortran, using JNA. (It's a numerical library, and lots of math people do their coding in Fortran.) It is compiled to a .so library, see below:

• Fortran: https://github.com/mizzao/libmao/tree/master/src/main/fortran
• Java binding: https://github.com/mizzao/libmao/blob/master/src/main/java/net/andrewmao/probability/MvnPackDirect.java

I was getting great results with everything unit tested in my code, but then I tried using the code from multiple threads, and everything started failing with strange errors. I then looked into some stuff about reentrant Fortran code and realized that the library I was using has the equivalent of some global variables (SAVE keywords in Fortran, which remember the values of variables when a function is called again: fortran SAVE statement)

For now I am wrapping calls to the library in synchronized blocks, but this is hobbling performance significantly. It seems to me that it would take significant effort to re-engineer the library to be reentrant (it has a few thousand lines of numerical code, and it's not clear how the values carry over when the subroutines are being run.) Does anyone know the best way to get around the problem? My imagination suggests...

• Is there some way to get each Java thread to load a separate copy of the shared library in memory, so that the global variables are effectively thread-local? Is that even possible? I'm not sure about how JNA's direct binding or library binding works, and if there's a way to use it that way.
• Would it still be screwed even if it were called from different VMs? How can I check to make sure?
• Is there some way to get gfortran (gcc) to compile the Fortran code in a way that is reentrant?
• Is there some quick and dirty way to make the Fortran code reentrant? I've searched about the RECURSIVE keyword, which apparently keeps variables on the stack, but that doesn't seem to be compatible with the existing code.
• Any other possible solutions?

I confirm that things are ok with multiple VMs; this makes sense as they don't share memory. Still a PITA and way more inconvenient than threads though.

Garfieldgarfinkel

What if you create N copies (with different names) of your library and load it N times? Kind of like a "library pool". Every thread will have to get a copy of the library and return it when done. You will still have to have some sort of synchronization for getting and returning the library. Something that will insert a RW fence. If you library loads other not safe libraries then this won't work. – Unscramble

@jdb, very creative approach! For now I'm running embarrassingly parallelizable code in different VMs, which is almost the same thing. Let's see what else turns up. – Garfieldgarfinkel

N copies would work (and would work with JNA as well (see Native.synchronizedLibrary()) provided that the library's usage of system resources is unique to its process (e.g. not always using a file called "/tmp/output.txt"). – Harner

Thanks @technomage. Just got around to implementing this; see below :) – Garfieldgarfinkel

Answer 1

For reference I just wanted to share the following class that I implemented for doing this. It takes a given library and interface, makes n copies and maps the JNA proxied interfaces to each copy, then returns another proxied interface that implements thread-safe locking, creating a version that is re-entrant and can run up to the number of processors one has.

public class LibraryReplicator<C> {

    final BlockingQueue<C> libQueue;
    final Class<C> interfaceClass;
    final C proxiedInterface;

    @SuppressWarnings("unchecked")
    public LibraryReplicator(URL libraryResource, Class<C> interfaceClass, int copies) throws IOException {
        if (!interfaceClass.isInterface()) 
            throw new RuntimeException(interfaceClass + "is not a valid interface to map to the library.");

        libQueue = new LinkedBlockingQueue<C>(copies);
        this.interfaceClass = interfaceClass;

        // Create copies of the file and map them to interfaces
        String orig = libraryResource.getFile();
        File origFile = new File(orig);
        for( int i = 0; i < copies; i++ ) {
            File copy = new File(orig + "." + i);
            Files.copy(origFile, copy);                     

            C libCopy = (C) Native.loadLibrary(copy.getPath(), interfaceClass);         
            libQueue.offer(libCopy); // This should never fail
        }               

        proxiedInterface = (C) Proxy.newProxyInstance(
                interfaceClass.getClassLoader(), 
                new Class[] { interfaceClass }, 
                new BlockingInvocationHandler());
    }

    public LibraryReplicator(URL libraryResource, Class<C> interfaceClass) throws IOException {
        this(libraryResource, interfaceClass, Runtime.getRuntime().availableProcessors());
    }

    public C getProxiedInterface() {
        return proxiedInterface;
    }

    /*
     * Invocation handler that uses the queue to grab locks and maintain thread safety.  
     */
    private class BlockingInvocationHandler implements InvocationHandler {
        @Override
        public Object invoke(Object proxy, Method method, Object[] args) throws Exception {
            C instance = null;

            // Grab a copy of the library out of the queue          
            do {
                try { instance = libQueue.take(); }
                catch(InterruptedException e) {}
            } while(instance == null);

            // Invoke the method
            Object result = method.invoke(instance, args);

            // Return the library to the queue
            while(true) {
                try { libQueue.put(instance); break; }
                catch( InterruptedException e ) {} 
            } 

            return result;
        }       
    }

}

An example usage is something like the following as part of a static initializer:

MvnPackGenz lib = new LibraryReplicator<MvnPackGenz>(
        MvnPackGenz.class.getClassLoader().getResource("mvnpack.so"), 
        MvnPackGenz.class).getProxiedInterface();

This creates a bunch of copies of the library (in my case, 12) creating the lib variable above which 'looks' re-entrant, and which can be safely run by multiple threads:

-rw-r--r-- 1 mao mao 50525 Sep 26 13:55 mvnpack.so
-rw-r--r-- 1 mao mao 50525 Sep 26 18:21 mvnpack.so.0
-rw-r--r-- 1 mao mao 50525 Sep 26 18:21 mvnpack.so.1
-rw-r--r-- 1 mao mao 50525 Sep 26 18:21 mvnpack.so.10
-rw-r--r-- 1 mao mao 50525 Sep 26 18:21 mvnpack.so.11
-rw-r--r-- 1 mao mao 50525 Sep 26 18:21 mvnpack.so.2
-rw-r--r-- 1 mao mao 50525 Sep 26 18:21 mvnpack.so.3
-rw-r--r-- 1 mao mao 50525 Sep 26 18:21 mvnpack.so.4
-rw-r--r-- 1 mao mao 50525 Sep 26 18:21 mvnpack.so.5
-rw-r--r-- 1 mao mao 50525 Sep 26 18:21 mvnpack.so.6
-rw-r--r-- 1 mao mao 50525 Sep 26 18:21 mvnpack.so.7
-rw-r--r-- 1 mao mao 50525 Sep 26 18:21 mvnpack.so.8
-rw-r--r-- 1 mao mao 50525 Sep 26 18:21 mvnpack.so.9

You can see an updated version at

https://github.com/mizzao/libmao/blob/master/src/main/java/net/andrewmao/misc/LibraryReplicator.java

Garfieldgarfinkel

Where in your github repository (or somewhere else) is the file "mvnpack.so"? I am trying to get your library running, but without the file I get errors. – Rhachis

There is a Makefile where you compile it on your machine: github.com/mizzao/libmao/blob/master/src/main/fortran/Makefile – Garfieldgarfinkel

Answer 2

I'm not sure about each thread having a separate instance of the library, however here's what I did a number of years ago: Get the operating system to make in re-entrant for you.

I ended up creating a pool of application instances on the Unix machine, and communicating with them using network sockets - each process was listening on its own socket.

Even if the library is not re-entrant, starting it as a separate process will be ok. . . Perhaps you can write a thin unix wrapper around the library and communicate over your own proprietary protocol.

Giglio