P/Invoke to dynamically loaded library on Mono

Question

I'm writing a cross-platform .NET library that uses some unmanaged code. In the static constructor of my class, the platform is detected and the appropriate unmanaged library is extracted from an embedded resource and saved to a temp directory, similar to the code given in another stackoverflow answer.

So that the library can be found when it isn't in the PATH, I explicitly load it after it is saved to the temp file. On windows, this works fine with LoadLibrary from kernel32.dll. I'm trying to do the same with dlopen on Linux, but I get a DllNotFoundException when it comes to loading the P/Invoke methods later on.

I have verified that the library "libindexfile.so" is successfully saved to the temp directory and that the call to dlopen succeeds. I delved into the mono source to try figure out what is going on, and I think it might boil down to whether or not a subsequent call to dlopen will just reuse a previously loaded library. (Of course assuming that my naïve swoop through the mono source drew the correct conclusions).

Here is the shape of what I'm trying to do:

// actual function that we're going to p/invoke to
[DllImport("indexfile")]
private static extern IntPtr openIndex(string pathname);

const int RTLD_NOW = 2; // for dlopen's flags
const int RTLD_GLOBAL = 8;

// its okay to have imports for the wrong platforms here
// because nothing will complain until I try to use the
// function
[DllImport("libdl.so")]
static extern IntPtr dlopen(string filename, int flags);

[DllImport("kernel32.dll")]
static extern IntPtr LoadLibrary(string filename);


static IndexFile()
{
    string libName = "";

    if (IsLinux)
        libName += "libindexfile.so";
    else
        libName += "indexfile.dll";

    // [snip] -- save embedded resource to temp dir

    IntPtr handle = IntPtr.Zero;

    if (IsLinux)
        handle = dlopen(libPath, RTLD_NOW|RTLD_GLOBAL);
    else
        handle = LoadLibrary(libPath);

    if (handle == IntPtr.Zero)
        throw new InvalidOperationException("Couldn't load the unmanaged library");
}


public IndexFile(String path)
{
    // P/Invoke to the unmanaged function
    // currently on Linux this throws a DllNotFoundException
    // works on Windows
    IntPtr ptr = openIndex(path);
}

---

Update:

It would appear that subsequent calls to LoadLibrary on windows look to see if a dll of the same name has already been loaded, and then uses that path. For example, in the following code, both calls to LoadLibrary will return a valid handle:

int _tmain(int argc, _TCHAR* argv[])
{
    LPCTSTR libpath = L"D:\\some\\path\\to\\library.dll";

    HMODULE handle1 = LoadLibrary(libpath);
    printf("Handle: %x\n", handle1);

    HMODULE handle2 = LoadLibrary(L"library.dll");
    printf("Handle: %x\n", handle2);

    return 0;
}

If the same is attempted with dlopen on Linux, the second call will fail, as it doesn't assume that a library with the same name will be at the same path. Is there any way round this?

Superorder

Answer 1

After much searching and head-scratching, I've discovered a solution. Full control can be exercised over the P/Invoke process by using dynamic P/Invoke to tell the runtime exactly where to find the code.

---

Edit:

Windows solution

You need these imports:

[DllImport("kernel32.dll")]
protected static extern IntPtr LoadLibrary(string filename);

[DllImport("kernel32.dll")]
protected static extern IntPtr GetProcAddress(IntPtr hModule, string procname);

The unmanaged library should be loaded by calling LoadLibrary:

IntPtr moduleHandle = LoadLibrary("path/to/library.dll");

Get a pointer to a function in the dll by calling GetProcAddress:

IntPtr ptr = GetProcAddress(moduleHandle, methodName);

Cast this ptr to a delegate of type TDelegate:

TDelegate func = Marshal.GetDelegateForFunctionPointer(
    ptr, typeof(TDelegate)) as TDelegate;

Linux Solution

Use these imports:

[DllImport("libdl.so")]
protected static extern IntPtr dlopen(string filename, int flags);

[DllImport("libdl.so")]
protected static extern IntPtr dlsym(IntPtr handle, string symbol);

const int RTLD_NOW = 2; // for dlopen's flags 

Load the library:

IntPtr moduleHandle = dlopen(modulePath, RTLD_NOW);

Get the function pointer:

IntPtr ptr = dlsym(moduleHandle, methodName);

Cast it to a delegate as before:

TDelegate func = Marshal.GetDelegateForFunctionPointer(
    ptr, typeof(TDelegate)) as TDelegate;

---

For a helper library that I wrote, see my GitHub.

Superorder

I would like to see this solution, but the document linked to seems related, but not complete. The linked document has two other links that might have been useful, but the links are now dead. It would have been nice to have the solution described directly instead of just a link. – Unsettled

Answer 2

Try running it like this from a terminal:

export MONO_LOG_LEVEL=debug
export MONO_LOG_MASK=dll
mono --debug yourapp.exe

Now every library lookup will be printed to the terminal, so you'll be able to find out what's going wrong.

Betwixt

I have already done this -- it doesn't tell me anything I don't already know. It tells me that the library can't be found... – Superorder

Try setting LD_LIBRARY_PATH to the directory where you're saving your libraries (before starting your program, so you'd do something like this: "LD_LIBRARY_PATH=/my/tmp/dir:$LD_LIBRARY_PATH mono myapp.exe" from a terminal). – Betwixt

Answer 3

I needed to load a native library extracted to a temporary location, and I almost found a solution. I've checked Mono's source code and figured out a way:

[DllImport("__Internal", CharSet = CharSet.Ansi)]
private static extern void mono_dllmap_insert(IntPtr assembly, string dll, string func, string tdll, string tfunc);

// and then somewhere:
mono_dllmap_insert(IntPtr.Zero, "somelib", null, "/path/to/libsomelib.so", null);

This kind of works. The problem is, you cannot allow Mono's stupid JIT compiler to catch a whiff of any DllImported method referring this library before calling mono_dllmap_insert().

Because if it does, strange things will happen:

Mono: DllImport searching in: '/tmp/yc1ja5g7.emu/libsomelib.so' ('/tmp/yc1ja5g7.emu/libsomelib.so').
Mono: Searching for 'someGreatFunc'.
Mono: Probing 'someGreatFunc'.
Mono: Found as 'someGreatFunc'.
Error. ex=System.DllNotFoundException: somelib

So now that I'm calling my native someGreatFunc(), Mono is able to find the library and load it (I checked), it is able to find the symbol (I checked), but because somewhen in the past when it was doing JIT it was not able to load that library, it decides to throw DllNotFoundException anyway. I guess the generated code contains a hardcoded throw statement or something :-O

When you call another native function from the same library that happens not to have been JITted before you called mono_dllmap_insert(), it will work.

So you can either use the manual solution added by @gordonmleigh or you must tell Mono where the library is BEFORE it JITs any of these imports. Reflection may help there.

Longish

Answer 4

Not sure why you think this is related to mono, since the issue you're having is not about mono's dynamic loading facilities.

If your updated sample works, it just means that LoadLibrary() on windows has different semantics than dlopen() on Linux: as such you either have to live with the difference or implement your own abstraction that deals with the directory issue (my hunch is that it's not the directory that is retained, but windows simply looks to see if a library with the same name was already loaded and it reuses that).

Lionize

It's related to mono in the sense that what I am trying to achieve is something on mono. There might be an easy solution that is specifically mono-related. Yes I agree with your hunch, sorry if that wasn't clear. Unfortunately I can't see how to write an abstraction, because mono uses dlopen directly when doing P/Invoke. – Superorder

This is not related to mono because you would have the exact same issue if you used C, C++, python or any other programming language, since the problem you're having is that dlopen() is not an exact match to LoadLibrary(). The abstraction you need is something like: handle = lookup_loaded_handle (basename (library_path)); if (handle != null) return handle; ekse return dlopen (library_path); – Lionize

This is related to mono because I'm not in charge of that abstraction. I can't rewrite the mono framework to change how it handles DllImport or the way it calls dlopen. As it turns out, I can execute unmanaged code without involving DllImport (see my answer). This solution was C# code; this remains a .NET/mono problem. – Superorder