Dispatcher's PriorityQueue causing "Memory Leaks"

Question

I am having issues with garbage collection in a UI Framework I've built in WPF. I use memory leak in quotes because I think I understand the problem, but do not have a solution or some sort of work around.

This issue occurs when I create UIElements but do NOT display them. From a UI perspective, I am using a Virtualized List View (GridView). So all of the Rows are not displayed at once. In one of my use cases, I created an Export Rows features that has the ability to export all of the rows to a csv. Usually, the cells within the row are comprised of primitives like strings or ints.

Where it gets tricky is some of the cells are UIElements themselves, which is one of the reasons the list is virtualized (the cost to make some is time consuming and can be a memory hog). So I only create the UIElements when asked and then I don't even cache them (to make them be eligible for collection).

The issue occurs when I access the properties while I am enumerating through the row provider. The UIElements are created and returned, the data extracted from them, written to csv, and then the next row is looked at. In this loop, there are no references to these Rows (ViewModel Classes), so one would think they would be eligible for garbage collection. This appears to not be the case. Using .Net Memory Profiler, I have deduced that my "Out of Memory Exceptions" are a result the UIElements being held up in memory by a DispatcherOperation in a PriorityQueue (from Dispatcher). I assume this is because they're waiting to be displayed (but never do).

If I don't get an out of memory exception, EVENTUALLY these UIElements appear to be processed through the PriorityQueue (I'm guessing it gives up) and are garbage collected. This is the best case scenario. This seems to be fine when I'm dealing with a small number of Rows. But when we get into the 50,000 - 100k level, it's another story. I can assure you there are no other references to these ViewModels or to the UIElements themselves (outside of the DispatcherOperation).

Any thoughts on how I can fix this or get around this problem? Is there anyway to block this queuing of these undisplayed, and soon to be unused UIElements?

Edit 01/25/2013: I realized I might need to clear up specifically what is saved in memory. The row objects, because they don't have a reference to the UIElement, which are not saved in memory (this is good). Only the UIElements that are accessed via a Get accessor are sticking around in memory. The only reference to them is the PriorityQueue and nothing of my own code.

Richly

You cannot bind the elements to a (data) data source? – Albina

I'm not sure what you're asking. The ViewModels that eventually get turned into ListViewItems by the ListView, are created and are programmatically binded when they come into view. There is no issues with that, its purely when I'm creating UIElements but not showing them (I don't want to I'm exporting them). – Richly

UIElement is pretty expensive, you don't need a leak to get OOM when you create a hundred thousand of them. Just don't do this, keep the model separated from the view. – Bournemouth

For the most part this is the case. I am actually extremely trying to keep this the case and do not want my exporting functions to know anything about "models". For most cases my view model does separate the view from the model.The problem is if I want to show something like multiple values in a single cell. The only way I've seen to do something like this is to either physically write the xaml and bind to it (not possible) or use cell templates, which I have yet to get to work correctly. If i just set the value of the cell to a combobox it works and everything is happy. – Richly

Is there perhaps another way that I may not know about that I can expose many values in a single cell without resorting to creating my own UIElements? In some cases I'm making ComboBoxes that contain Grids (3 columns 1 row) in them. – Richly

any final solution with full source code ? – Persevere

Nope not really, I ended up going an entirely different route. I completely stripped all UIElement types out of my cells/rows. I could never get beyond this, so I just made the decision to handle a handful of data types to include a few "complex" types (a list would become a listbox) and so on. – Richly

After 7 years this has now got a bounty on it? I don't understand why you're not using binding with data templates. Now you can determine how many elements you want to hold in memory for virtualisation. Instead of creating new elements I would recycle items. Can you tell me why you are doing this manually? – Watchman

@flayn You really should open your own question and provide some implementation details along with it. WPF controls are designed with MVVM in mind: they usually don't expose the item containers (UIElement), but the data items, that are wrapped by the containers. A control separates view (how data is rendered or interacted with) from data models. UIElement is designed to render the data. It is not designed to handle data. It makes no sense to create UIElement instances in order to get access to data. If this is the case, then you definitely are doing everything wrong (lead by a wrong mindset). – Athalla

Without seeing any code for how the UIElements are created/destroyed, it'd be hard to help answer this. – Pyroelectricity

I have a suspiction: @Mike, you say "..., the UIElements are created and returned, the data extracted from them, written to csv, ..."; can I see the code you use to write to CSV? – Antimagnetic

Answer 1

Alright, I'm not a garbage collection expert, but I know two things:

1. You almost never have to care about garbage collection
2. You almost never have to do garbage collection manually

Those are the reasons why I don't know much about it, and I don't need to know much about it.

So besides that, here's a rough overview how garbage collection can work.

Imagine having this code:

class Program
{
    class Something
    {
        public string name { get; set; }
    }


    class Container
    {
        List<Something> myList = new List<Something>();

        public void AddNewSomething()
        {
            Something mySomething = new Something() { name = "test" };
            myList.Add(mySomething);
        }
    }

    public static void Main(string[] args)
    {
        Container myContainer = new Container();
        myContainer.AddNewSomething();

        while (true)
        {
            Console.WriteLine("Something will always be in Memory");
        }
    }
}

Garbage collection is all about objects being referenced. So look at the method: AddNewSomething. Is the local variable mySomething required after the method is finished? Can that object be garbage collected? The answer is no, because the object mySomething is now referenced by myList.

Since there's an endless loop in Main myList cannot be garbage collected, and since it contains an object of MySomething, that object cannot be garbage collected either, because otherwise it would disappear from the list.

Makes sense? I hope so.

Given UIElements, I believe they always have a parent? A container which holds them. That parent is usually an object having a list of children, like in my example. So unless the parent cannot be disposed by the garbage collector since there's a reference to it (It needs to be displayed in the window for example), the children cannot be disposed by the garbage collector. So the list of children grows bigger and bigger the more UIElements you add. You have to actively remove them from the parent. There's some code missing, so I can't give a clear example, but you're looking for something like that:

someParent.Children.Remove(noLongerRequiredUiElement);

But besides all of that, why do you create and add UIElements you never show? It seems like your just dealing with some data, that can reside in a separate class which is not related to UI at all. It's not entirely clear to me why you do it this way, given your question. But I think you might think about your class architecture again. That way you can have items in a Queue, and remove them one by one as soon as they are processed.

Distemper

The problem is, that the UIElements get created but never displayed. This is due to some code that I cannot change (now). "The UIElements being held up in memory by a DispatcherOperation in a PriorityQueue (from Dispatcher)." – Plenipotentiary