Memory Overflow: Having an increasing number of Microsoft.CSharp.RuntimeBinder.Semantics

Question

We are currently hunting some memory leaks in our application, when doing some operation(loading and closing one project inside our application), we know that the memory increase always a little bit.

We have already found a lot of them, but now, the 10+ most increasing classes are (according to our tool, ANTS Memory Profiler 8.2):

• Microsoft.CSharp.RuntimeBinder.Semantics.SYMTBL+Key
• Microsoft.CSharp.RuntimeBinder.Semantics.LocalVariableSymbol
• Microsoft.CSharp.RuntimeBinder.Semantics.CONSTVAL
• Microsoft.CSharp.RuntimeBinder.Semantics.EXPRCONSTANT
• Microsoft.CSharp.RuntimeBinder.Semantics.EXPRCLASS
• Microsoft.CSharp.RuntimeBinder.Semantics.EXPRTYPEOF
• Microsoft.CSharp.RuntimeBinder.Semantics.EXPRLIST
• Microsoft.CSharp.RuntimeBinder.Semantics.MethWithInst
• Microsoft.CSharp.RuntimeBinder.Semantics.CMemberLookupResults
• Microsoft.CSharp.RuntimeBinder.Semantics.EXPRMEMGRP
• Microsoft.CSharp.RuntimeBinder.Semantics.EXPRCALL
• Microsoft.CSharp.RuntimeBinder.Semantics.EXPRWRAP
• Microsoft.CSharp.RuntimeBinder.Semantics.AggregateDeclaration
• Microsoft.CSharp.RuntimeBinder.Semantics.Scope

Unfortunately, I don't what this is, so It's a little bit hard for me to find how/what I should release.

I checked the instance tree but, it goes all the way with microsoft stuff.

The issue is that when we do the "Open/close" of a project, we go through a lot(most of) our code.

EDIT One part of our application uses the dynamic keyword for some resources, it may be linked. The class here are not Disposable, should I do something special with them?

EDIT 2

I'm pretty sure this is related to my dynamic stuff, it seems that C# create a cache when using dynamic. But currently I've no idea why it grows(I load the same classes all the time, and I will have exactly the same signature all the time), nor how to clear this.

Spritsail

Is there anything you found about this topic? – Drama

@MarChr Currently not :( I'm pretty sure it has something to do with my usage of dynamic, but I cannot understand why this is continuously growing – Spritsail

Hmm.. Seems like i ran into the same issue. But it's a huge application, hard to figure out where it's going wrong. But when i find something out i post an example to let you know! – Drama

@MarChr Any chance you found a solution? – Spritsail

J4N, is it safe to assume that you have a leak of a dynamic object of yours, which manifests itself via the CSharp.RuntimeBinder.Semantics stuff because that stuff represents much more memory than your dynamic objects themselves? Can we safely say that your problem can be fixed by finding and fixing a memory leak in your code? In other words, could it be that the right answer would be an answer that would aid in fixing memory leaks in general, regardless of your particular situation? – Mitsukomitt

@MikeNakis No, I've already fixed a lot of MemoryOverflow in the past. The issue here is that I don't create myself the "Semantics" objects. I also tried to replace all my dynamic by object and doing some dirty typecasting+ reflection and I've no more memory leaks issue at all. So the issue seems to be really related to the dynamic cache – Spritsail

It may very well be so, but experience tells us that when someone comes to stackoverflow with behaviour which indicates a bug in the framework, in 99.9% of the cases the bug turns out to be in their code. So, even though I have every intention to believe you, the prudent approach on my behalf is to nonetheless try to find a flaw in your code. – Mitsukomitt

I don't say it's a bug in the framework, but probably my usage(my assumption is that I use a lof dynamic in within LINQ request, so anonymous delegate, and I don't know how well .Net handle them when having to store for which "call site" it has to cache those data – Spritsail

Do you really need to use dynamic? Can you pinpoint a code snippet in your application where dynamic is being used and post it here? – Pericycle

There is a section about Dynamic in the book 'Writing High-Performance .NET Code'. This is what the author has to say about 'dynamic' at the beginning of the section: "Any code using the dynamic keyword, or the DLR, is not going to be highly optimized. Performance tuning is often about stripping away abstractions but using DLR is adding one huge abstraction layer". – Experimental

Answer 1

I encountered exactly the same issue today by profiling memory leaks in my app RepoZ. That tool is supposed to run in the background, checking Git repositories and update Windows Explorer window titles periodically. The latter task has to make some COM calls to "Shell.Application" to find the Explorer windows and determine the path they are currently pointing to.

By using the dynamic keyword like this ...

dynamic shell = Activator.CreateInstance(...);
foreach (object window in shell.Windows())
{ 
    var hwnd = window.Hwnd;
    ...
}

... I ended up to a memory dump like that after a few hours:

Combridge

To solve that, I wrote a little helper class called "Combridge" caring to release COM objects and providing quite easy access to methods and properties of the underlying COM object. It is very easy and straightforward, nothing special here. It makes use of Reflection to COM objects, that's why there's some loss in performance (see below).

With it, the code sample from above looks like this:

using (var shell = new Combridge(Activator.CreateInstance(...)))
{
    var windows = shell.InvokeMethod<IEnumerable>("Windows");
    foreach (var window in windows)
    {
        var hwnd = window.GetPropertyValue<long>("Hwnd");
        ... 
    }
}

You can see the file ExplorerWindowActor on how it is used in RepoZ.

It is not exactly as beautiful as with dynamic and performance got worse in this first attempt as well. A quick bench showed the following:

Performance

I tested 1000 iterations, in each iteration 10 open Explorer windows were processed. For each window, 4 methods or properties are invoked on that COM object. So we're talking about 40.000 COM calls.

The duration went up from ~2500ms (dynamic) to ~6000ms (Combridge). That's from 0.062ms to 0.150ms for each call.

So this takes about 2.4x the time to finish now.

This is significant, I know. But it is okay for my requirements and the memory leak is gone.

That's it - I wanted to share that story with you, hopefully you can use that class (or an improved version of it) to get out of the dynamic hell as well.

~Update~

After 10 hours, RepoZ still runs with a very constant memory footprint.

So with 10 Explorer windows open, 4 COM calls per window and that whole loop two times a second, RepoZ created about 72.000 COM instances and made about 2.880.000 COM calls in total without any increase in memory consumption.

I guess we can say that the issue really comes with dynamic.

Waadt

Answer 2

Dynamic keyword should be used seldom since in most of the cases workarounds can be found not requiring it.

Based on your application, the best advise is to carefully think if you can design your solution so that you avoid dynamic. Here are some valid use cases for dynamic: https://msdn.microsoft.com/en-us/library/dd264736.aspx

Given that you really need to use dynamic, I would suggest instrumenting your code and figure out what parts are the most memory consuming. Indeed using dynamic increases your memory consumption based on the fact that it needs to perform all kinds of lookups, but to have a out-of-memory exception you would need to use a lot of dynamic variables for a lot of unknown types.

There are a lot of different ways for calling methods on unknown types, and measuring and tuning the bottlenecks is the way to go.

PS: Also, posting some code snippets helps a lot.

Pericycle