Force unloading of DLL from assembly

Question

I am attempting to unload a misbehaving third-party DLL from my .NET process, as it seems to be causing a problem which is always resolved by restarting my application. Rather than restarting the application, I'd like to remove and reload the DLL instead.

The DLL is being loaded using LoadLibrary and removed using FreeLibrary (using DllImports taken from the P/Invoke website). When I call LoadLibrary() I see the DLL appear in the list of DLLs in Process Explorer, and when I call FreeLibrary() I see the DLL disappear from the list of DLLs - as expected.

However, once I have called the Initialize() function of the third-party library, FreeLibrary() no longer removes the DLL from the list, even if I call a corresponding Deinit() method beforehand. Calling another function in the library does not have this problem. However, I must Initialise() the library before use!

I have tried isolating the DLL by creating it in its own AppDomain, then unloading this domain after the DLL is freed.

I get no error return codes or exceptions from Initialize() or Deinit(), from the LoadLibrary() or FreeLibrary() or from creating or unloading the AppDomain.

I used the following code to create the AppDomain and initialise:

string pathToDll = Assembly.GetExecutingAssembly().CodeBase;
m_Domain = AppDomain.CreateDomain("MyAppDomain", null, new AppDomainSetup { PrivateBinPath = pathToDll });
m_Module = (ThirdPartyModule)m_Domain.CreateInstanceFromAndUnwrap(pathToDll, typeof(ThirdPartyModule).FullName);
m_Module.Init();

To deinitialise and unload the AppDomain:

m_Module.Free();
m_Module = null;
if (m_Domain != null)
{
    AppDomain.Unload(m_Domain);
    m_Domain = null;
}

Finally, my ThirdPartyModule assembly class:

internal class ThirdPartyModule : MarshalByRefObject
{
    [DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
    internal static extern IntPtr LoadLibrary(string lpFileName);

    [DllImport("kernel32.dll", SetLastError = true)]
    [return: MarshalAs(UnmanagedType.Bool)]
    internal static extern bool FreeLibrary(IntPtr hModule);

    public IntPtr Module { get; set; }

    public ThirdPartyModule()
    {
        Module = LoadLibrary("Misbehaving.dll");
    }

    public void Free()
    {
        FreeLibrary(Module);
        Module = IntPtr.Zero;
    }

    // ...
}

Does this look like it should behave as I expected? If not, is there any other way I can ensure that this DLL is totally unloaded by my process?

Edit: More info

• The DLL contains native code, likely compiled from C/C++
• Unfortunately my process is constrained to using .NET 2 only (so no WCF solution).
• I am using WinXP Pro x64 SP2 but the solution must be XP, Win7 x32/x64 etc compatible.
• The DLL is used to communicate with a USB token

Otes

Could you run the DLL in another process and communicate with it through WCF. You can simply kill the entire process when you wan't to unload it. – Davisdavison

@Yahia - yes, I think it's a C/C++ DLL. – Otes

@Davisdavison - Nice idea, but I can only use .NET 2 :(. Updated question to reflect this. – Otes

even in .NET 2.0 you could still load the dll in another appDomain and use available inter-domain communication methods then kill the second appDomain when you want to unload the dll. – Nephograph

Unless you use @CodingBarfield's idea but use instead of WCF use HttpListener / HttpListenerRequest / HttpListenerResponse to communicate with the process. It kind of becomes a little mini web server which you can GET and POST against. – Arborescent

@gt which Windows version are you running on ? – Eley

@Eley WinXP Pro x64 SP2 – Otes

Win32 knows nothing about AppDomains, so AppDomains aren't going to help with a native dll. Put the library-facing code into a separate process, choose an appropriate IPC method and you're set. IPC doesn't have to be WCF, you can use HttpListener as JohnL suggests or shared memory or mailslots or pipes or plain sockets. The choice depends on how your program interacts with the native dll, what its input/output consists of etc. – Icono

@gt is the desktop fireall active by any chance ? what exactly does that DLL do ? – Eley

You might want to try calling FreeLibrary(Module) in a loop until it fails. Windows keeps a loaded count for DLLs and only when it goes to zero does it unload the DLL. – Nous

@500 — that's just papering over the problem. FreeLibrary doesn't unload that dll on the first go for a reason. Forcing an unload like that may leak resources or even set the process up for a segfault (e.g. if the dll hooked other dlls' functions). When it's in a process by itself, at least the kernel will release kernel resources and other processes' memory won't be corrupted. – Icono

Answer 1

I would recommend to implement a separate process (EXE) which your application launches and which in turn loads the DLL.

This allows you to kill the process whenever need be...

I see several options on how to communicate - for example:

• you could use COM (if you implement it as an out-of-process COM server)
• you could use shared memory (very high performance, see this for a walkthrough and this for a .NET 2 wrapper)

Since you write that the method must be compatible with several Windows versions and some come with a desktop firewall I would refrain from using anything "networky" for the IPC.

Eley