Sharing data between AppDomains

Question

I have a process that can have multiple AppDomains. Each AppDomain collect some statistics. After a specified time, I want to accumulate these statistic and save them into a file.

One way to do this is Remoting, which I want to avoid.

The only other technique I have in mind is to save each AppDomain's data in a file, and after a specific time, one of the AppDomain collects all data and accumulate them.

But it would be ideal if this all could be done in-memory, without the cost of serializing the information to pass between AppDomains. Anyone have any ideas?

Smear

Answer 1

The only way to avoid serialisation is to represent your data using objects which derive from MarshalByRefObject, but in that case you will still have the cost of marshalling across the AppDomain boundaries. This may also involve the refactoring/re-writing of much of your code.

Assuming marshalling by reference is not an option, you will have to serialise at some point. It simply cannot be avoided. One way to do this is as Neil Barnwell suggests, with a database, another would be with a local file as you suggest yourself.

Another way which may or may not feasible depending on your delivery timeline and/or .NET 4.0 adoption, would be to use a memory mapped file, see .Net Framework 4.0: Using memory mapped files.

Lempira

I have not written code yet. Just working on the design. Can you tell me of any article that explains the sharing of data using the first approach you posted? – Smear

Marshaling by reference will serialize the data, too, but in small pieces. Every method call will return a bit of information effectively serializing a bit of the data. This is probably a good idea if you require only a small portion of the data. But if you have to process (almost) the whole data, getting it bit by bit with many cross domain calls will be incredible slow compared to serializing and transferring the data at once. – Tanah

If you follow this road, don't forget to override InitializeLifetimeService method; that was driving me crazy a few days ago ("Object '...' has been disconnected or does not exist at the server.") – Lyndell

Answer 2

It is possible to share data between AppDomains without the costs of Marshalling. But it is a rather hacky way. You can create a source data object which is shared by reference between all AppDomains. This way you get all data into one shared object without the costs of Marshalling. Sounds too easy to be true?

The first thing is to know how share data between AppDomains without Marshalling. For this you get the object address of your data source object via Marshal.UnsafeAddrOfPinnedArrayElement. Then you pass this IntPtr to all AppDomains which are interested in this. In the target AppDomain you need to cast this IntPtr back to an object reference which can be done JIT::CastAny which is done if you return an object from a method and push the pointer of it onto the stack.

Viola you have shared an object as plain pointer between AppDomains and you get InvalidCastExceptions. The problem is that you must set for all your AppDomains LoaderOptimization.MultiDomain to ensure that the assembly that defines the shared data type is loaded as AppDomain neutral type which has the same Method Table pointer between all AppDomains.

You can find an example application that does exactly this as part of WMemoryProfiler. See this link for a more detailed explanation and download link to the sample code.

The basic code is

[LoaderOptimization(LoaderOptimization.MultiDomain)]
static public void Main(string[] args)
{

    // To load our assembly appdomain neutral we need to use MultiDomain on our hosting and child domain
    // If not we would get different Method tables for the same types which would result in InvalidCastExceptions
    // for the same type.
    var other = AppDomain.CreateDomain("Test"+i.ToString(), AppDomain.CurrentDomain.Evidence, new AppDomainSetup
        {
            LoaderOptimization = LoaderOptimization.MultiDomain,
        });

    // Create gate object in other appdomain
    DomainGate gate = (DomainGate)other.CreateInstanceAndUnwrap(Assembly.GetExecutingAssembly().FullName, typeof(DomainGate).FullName);

    // now lets create some data
    CrossDomainData data = new CrossDomainData();
    data.Input = Enumerable.Range(0, 10).ToList();

    // process it in other AppDomain
    DomainGate.Send(gate, data);

    // Display result calculated in other AppDomain
    Console.WriteLine("Calculation in other AppDomain got: {0}", data.Aggregate);
    }
}

Fleecy

Really nice answer. I was first in doubt whether sharing the same objects between two AppDomains like this would wreak havoc with GC cause of any unpinned objects (only one shared object is pinned). But as you nicely outline in your article there is only one GC across all AppDomains, so it works out. Cool stuff! – Potbellied

Dropping to unmanaged code to try to work around intentional safety checks in the framework is a really excellent way to build an unmaintainable, cluster. There is a good reason that appdomains are seperate, both for security, and code safety, and MS clearly indicates that you either MarshallByRef or serialize when needing to pass between. This is a bad way to go, riddled with bad ends. This might be acceptable for extremely performance sensitive code, extremely contained code, but an awful general suggestion for anyone else. Threading and async will wreak havoc with this. – Pytlik

@Tim: I do not hope anyone is using this except for educational purposes. But since all are going after .NET Core which has no AppDomain support this answer will not be relevant anymore anyway since .NET Core has much more to offer in terms of performance. – Fleecy

Very true. Core will be so much nicer as it gets polished – Pytlik

@Pytlik AppDomains are not always used for security. Sometimes they are a means of allowing for assembly unloading. The moment you drop to unsafe code, any notions of safety are out the window, that much should be clear to anyone. – Renn

Answer 3

I tend to say just use remoting. Writing the data to a file requires serialization, too. Serialization seems to be almost unavoidable what ever technology you use. You have to transfer data from one application domain to another using some channel and you will have to serialize the data in order to get it through the channel.

The only way to avoid serialization seems to be using shared memory so that both application domains can access the data without ever going through a channel. Even deep cloning the data from one application domain's memory into the other's memory is at its core nothing more then a binary serialization (where the result is not necessarily stored in consecutive memory locations).

Elbertina

Remoting also involve Reflection. That is Serialization + Reflection. On the other hand my data is just some long and double values that I can write in file without much overhead. – Smear

You are looking at the wrong spots. The bottleneck of using a file is the disc access and will take several milliseconds and the transfer rate letting you transfer below one hundred megabyte per second. I am not sure what the actual bottleneck of remoting is (as far as I remember the performance is limited by the number of cross domain calls, not the amount of transfered data) but it is possible to transfer several hundred megabyte per second between application domains. Remoting strings using the fast path achieves transfer rates of several gigabyte per second. – Tanah

Answer 4

I do appreciate you want to keep this in-memory, but my first suggestion would be to write the data to a database and query from there. Remoting is still a remote call, which is where much of the "cost" of using a database server comes from, and you'd have to build in transaction-handling to make sure you don't lose data. If you write to a SQL Server database you have transaction support ready and waiting for you, and it's fast-fast-fast for queries.

Borrowing

While it may be a good idea to use a database and have the data persisted and solved to communication problem with an established technology, I do not think transactions would be a key benefit. If the source application domains crashes, the data is lost no matter if it just was on the wire to the database or in an in memory stream. – Tanah