AppDomains are a pure managed code construct. Nothing like that exists in native code, nor does Windows have any idea about it. So the scope for a loaded native DLL is the process. Technically, the pinvoke marshaller could reference count the DLL and keep track exactly which AppDomain triggered the load of the DLL. It however cannot tell whether any native code is running that uses that DLL. Native code that could be started by a call made from code in another AppDomain, possibly indirectly through a marshaled delegate.
Clearly disaster strikes if the AppDomain manager unloads a DLL that's used that way, that's a nasty and impossible to diagnose AccessViolation. Particularly nasty since it can trigger a long time after the AppDomain got unloaded.
So the marshaller doesn't implement that kind of counting, the DLL stays loaded. Only you can provide the guarantee that this can't happen, you have some measure of control over exactly what code runs in the DLL and how it gets started. You can force the DLL to unload but it requires a hack. Pinvoke LoadLibrary() yourself to get a handle to the DLL. And pinvoke FreeLibrary() twice to get it unloaded forcibly. Neither Windows nor the CLR can't see that you are cheating. You must ensure that the DLL can't be used after this.
