Manage to unmanaged transition from second AppDomain is very slow

Question

The setup is as follows:

• Main app domain loads a number of unmanaged C++ libraries from a C++/CLI assembly.
• A second app domain loads those C++ libraries, but of course it is just getting hold of handles to the previously loaded libraries.
• I run exactly the same managed C# code in both app domains that makes use of the unmanaged C++ libraries.
• The C++/CLI assembly contains a mixture of managed and unmanged code.

When calling into the unmanaged C++ libraries from the main app domain the cost of the managed to unmanaged transition is fairly negligible compared to the overall cost of execution. However, from the second app domain that did not originally load the libraries the cost of the managed to unmanaged transition becomes substantial; orders of magnitude larger. I know this because I have run ANTS performance profiler and this is what it tell me.

There are some unmanaged static variables in the C++/CLI assembly, I've tried replacing them when a new app domain is created, but this doesn't change the performance. There would well be a bunch of hidden ones.

What could be going on here? Why should the transition in the other app domain be so slow? Is there anything obvious to try to improve things?

Some context: the code is running in a separate app domain for isolation and so that it can be torn down when things go wrong. It would be very good to keep it isolated from the main app domain. It would be possible to start a new process from which to run the unmanaged code, however this could be expensive as it may require a lot of data to be serialized and deserialized and the cost of bootstrapping everything in a new process is also fairly large. In the general case the cost of the serialization and the bootstrap will be much lower than the cost of the managed to unmanaged transition that I'm seeing, but there is a lot of messing around involved and it still slows things down a lot.

Denticulation

Is your question in any way related to this one? In any case, a minimal reproducible example would be useful there. – Cower

It's similar, but the problem I'm getting is the other way around: I get the slow down going from managed to unmanaged memory. Yes I agree an MCV example would be great! I will see if I can hack something out. – Denticulation

I believe this is all expected. AppDomains bring you isolation, not performance. This isolation cost can happen anytime, but you'll pay for it anyway. C++/CLI is managed, even if you use some unmanaged code in it. Also see that one : #1144959 – Ulibarri

It would be expected if I had managed code in different AppDomains where one was calling the other. However I have native code that supposedly knows nothing about AppDomains. All I'm doing is calling that native code; why should it matter where it gets called from? Now it seems like it does matter; but I don't think it is obvious that it should. C++/CLI contains managed and unmanaged code. The unmanaged statics are not partitioned by app domain and I think that might be part of the problem. But it definitely isn't obvious. – Denticulation

I've tried putting together a simple solution that calls into native code and has statics and there is no difference in performance between app domains. So there must be something else going on that is causing something to get marshalled. – Denticulation

Have you tried stepping through the disassembly to gather clues? If you're paying for a cross-AppDomain call you may see UM2MDoADCallBack like I do which appears to be called by WrongAppDomain. It's not the answer, but it may help you narrow things down. – Dipterocarpaceous

@Dipterocarpaceous thanks for the tip. No I haven't, how do I do this? – Denticulation

Put a breakpoint on the call from managed to unmanaged code. Right click that line and choose "Go To Disassembly". With focus on the disassembly window use Step Into and follow execution all the way through and see what it's doing. You'll need to enable the Microsoft Symbol Server to get some helpful labels in the disassembly. You may have to change the Debugger Type setting to be able to step into the interop code. Look for call instructions to get an idea of what is going on. Those should have names and you can skip the obvious ones like GetThread. jmps matter too sometimes. – Dipterocarpaceous

OK, I'll give it a go, thanks. I've been trying to reproduce the issue in a toy example, but it seems to be pretty hard to do. I'm guessing it has something to do with statics and GCHandles, but still nothing concrete so far. – Denticulation

Just a thought, maybe there's a difference in security settings between the two appdomains and the slow-down comes from Demand checks for UnmanagedCode permission running on every call? – Sathrum

That's a good suggestion, however the settings for the second domain are inherited directly from the first, so there isn't any difference there. – Denticulation