Load Native Library from Class path

Question

I have a project setup that follows the Standard Directory Layout (not using Maven though):

src/main
      | java
      | resources
         | library.dll

Native DLLs are located in the resources folder and sources in the java folder. The resources folder is a member of the Java class path.

I would now like to load a DLL without having to set the JRE -Djava.library.path option or setting the PATH variable so the resulting jar file can be started with a simple double click.

Is it possible to add the resource folder to the library search path without having to do additional configuration when running the jar file? E.g. with a setting similar to the Class-Path in the Manifest?

Redhot

Answer 1

There is an old-time hack that still works as of today ( 1.7.0_55 & 1.8.0_05 ) to allow you to do to a runtime update using System.setProperty() and have the JVM notice the change. In general, you do the following:

System.setProperty("java.library.path", yourPath);
Field sysPath = ClassLoader.class.getDeclaredField( "sys_paths" );
sysPath.setAccessible( true );
sysPath.set( null, null );
System.loadLibrary(libraryName);

Google java sys_paths and look for articles about this technique.

Take care to handle errors/exceptions. Restore original paths as needed.

Cestode

Unfortunately, this does not seem to work with Java 1.8 – Redhot

@Mash - Did you also test at 1.7 or only 1.8? I upgraded one of my environments to 1.8.0_05 and all works fine to do a runtime modify of java.library.path using sysPath.set(null,null). Take a 2nd look at your impl. – Cestode

I actually can't reproduce it - in the process of generating a MWE, I found that if I create a new module (using IntelliJ) and copy all the contents of the old, erroneous module in it, it works. Although the modules are identical. It even works without any additional computing, IntelliJ now seems to take care of it properly. I'm sorry about that but I guess it was some kind of IDE bug. – Redhot

Answer 2

I have done something very similar in JNativeHook, you will need some helper code to determine the correct arch and os to load code for (See NativeSystem Class)

// The following code covered under the GNU Lesser General Public License v3.
static {
    String libName = System.getProperty("jnativehook.lib.name", "JNativeHook");

    try {
        // Try to load the native library assuming the java.library.path was
        // set correctly at launch.
        System.loadLibrary(libName);
    }
    catch (UnsatisfiedLinkError linkError) {
        // Get the package name for the GlobalScreen.
        String basePackage = GlobalScreen.class.getPackage().getName().replace('.', '/');

        // Compile the resource path for the native lib.
        StringBuilder libResourcePath = new StringBuilder("/");
        libResourcePath.append(basePackage).append("/lib/");
        libResourcePath.append(NativeSystem.getFamily()).append('/');
        libResourcePath.append(NativeSystem.getArchitecture()).append('/');


        // Get what the system "thinks" the library name should be.
        String libNativeName = System.mapLibraryName(libName);
        // Hack for OS X JRE 1.6 and earlier.
        libNativeName = libNativeName.replaceAll("\\.jnilib$", "\\.dylib");

        // Slice up the library name.
        int i = libNativeName.lastIndexOf('.');
        String libNativePrefix = libNativeName.substring(0, i) + '-';
        String libNativeSuffix = libNativeName.substring(i);
        String libNativeVersion = null;

        // This may return null in some circumstances.
        InputStream libInputStream = GlobalScreen.class.getResourceAsStream(libResourcePath.toString().toLowerCase(Locale.English) + libNativeName);
        if (libInputStream != null) {
            try {
                // Try and load the Jar manifest as a resource stream.
                URL jarFile = GlobalScreen.class.getProtectionDomain().getCodeSource().getLocation();
                JarInputStream jarInputStream = new JarInputStream(jarFile.openStream());

                // Try and extract a version string from the Manifest.
                Manifest manifest = jarInputStream.getManifest();
                if (manifest != null) {
                    Attributes attributes = manifest.getAttributes(basePackage);

                    if (attributes != null) {
                        String version = attributes.getValue("Specification-Version");
                        String revision = attributes.getValue("Implementation-Version");

                        libNativeVersion = version + '.' + revision;
                    }
                    else {
                        Logger.getLogger(GlobalScreen.class.getPackage().getName()).warning("Invalid library manifest!\n");
                    }
                }
                else {
                    Logger.getLogger(GlobalScreen.class.getPackage().getName()).warning("Cannot find library manifest!\n");
                }
            }
            catch (IOException e) {
                Logger.getLogger(GlobalScreen.class.getPackage().getName()).severe(e.getMessage());
            }


            try {
                // The temp file for this instance of the library.
                File libFile;

                // If we were unable to extract a library version from the manifest.
                if (libNativeVersion != null) {
                    libFile = new File(System.getProperty("java.io.tmpdir"), libNativePrefix + libNativeVersion + libNativeSuffix);
                }
                else {
                    libFile = File.createTempFile(libNativePrefix, libNativeSuffix);
                }

                byte[] buffer = new byte[4 * 1024];
                int size;

                // Check and see if a copy of the native lib already exists.
                FileOutputStream libOutputStream = new FileOutputStream(libFile);

                // Setup a digest...
                MessageDigest sha1 = MessageDigest.getInstance("SHA-1");
                DigestInputStream digestInputStream = new DigestInputStream(libInputStream, sha1);

                // Read from the digest stream and write to the file steam.
                while ((size = digestInputStream.read(buffer)) != -1) {
                    libOutputStream.write(buffer, 0, size);
                }

                // Close all the streams.
                digestInputStream.close();
                libInputStream.close();
                libOutputStream.close();

                // Convert the digest from byte[] to hex string.
                String sha1Sum = new BigInteger(1, sha1.digest()).toString(16).toUpperCase();
                if (libNativeVersion == null) {
                    // Use the sha1 sum as a version finger print.
                    libNativeVersion = sha1Sum;

                    // Better late than never.
                    File newFile = new File(System.getProperty("java.io.tmpdir"), libNativePrefix + libNativeVersion + libNativeSuffix);
                    if (libFile.renameTo(newFile)) {
                        libFile = newFile;
                    }
                }

                // Set the library version property.
                System.setProperty("jnativehook.lib.version", libNativeVersion);

                // Load the native library.
                System.load(libFile.getPath());

                Logger.getLogger(GlobalScreen.class.getPackage().getName())
                        .info("Library extracted successfully: " + libFile.getPath() + " (0x" + sha1Sum + ").\n");
            }
            catch (IOException e) {
                throw new RuntimeException(e.getMessage(), e);
            }
            catch (NoSuchAlgorithmException e) {
                throw new RuntimeException(e.getMessage(), e);
            }
        }
        else {
            Logger.getLogger(GlobalScreen.class.getPackage().getName())
                    .severe("Unable to extract the native library " + libResourcePath.toString().toLowerCase(Locale.English) + libNativeName + "!\n");

            throw new UnsatisfiedLinkError();
        }
    }
}

Nantucket

This still assumes a hard-coded resource path with libResourcePath = "/org/jnativehook/lib/" - is something as dynamic as the class path possible? I.e. Eclipse allows you to specify a folder as Class-Folder which is apparently added to the search-path. Something similar for the Manifest would be interesting. – Redhot

That would be a dynamic package name and its possible if you iterate though the entire package tree looking for a particular file name. That file name will need to be operating system specific and the search could take a long time. (Ex: x86_64_windows.bin) See: #177027 and #1811114 – Nantucket

Your NativeSystem class has a bug. Turkish I problem. Your toString() method should call lowercase with a fixed Locale. Otherwise "Unable to extract the native library /org/jnativehook/lib/ wındows /x86/JNativeHook.dll!" Notice the lowercase Turkish I, which is "ı" instead of "i" – Vulgar

Answer 3

I wanted to offer an alternative solution, because @Java42's answer is not portable (JVM dependant) and doesn't work starting from Oracle's/OpenJDK's Java 12.

You should use your own custom ClassLoader implementation. In the ClassLoader there's a method findLibary(String libname). This method returns the full path to the library to load. This can be used to load libraries in arbitrary places.

public class MyClassLoader extends ClassLoader {
    @Override
    protected String findLibrary(String libname) {
        return "/actual/path/to/library.so";
        // or return null if unknown, then the path will be searched
    }
}

Next step is to have you custom ClassLoader used by the JVM. So set it very soon in your code as the current threads contextClassLoader:

    public static void main(String[] args) {
        Thread.currentThread().setContextClassLoader(new StarterClassLoader());
        // ... your code
    }

Logsdon