WPF's ICollectionView.filter with large sets of data

Question

I'm working on an wpf app which contains a listview with quite a lot of data (10 000 to 100 000) rows. The user can apply all sorts of filters to this listview making the filter logic quite advanced (and slow). For now, the relevant part of my code looks like this:

ICollectionView view = CollectionViewSource.GetDefaultView(hugeList.ItemsSource);
view.Filter = new Predicate<object>(FilterCallback);

private bool FilterCallback(object item)
{
  //Filter logic
}

But this runs in the UI thread and blocks the entire application when filtering which gives a very poor user experience. So my question to you all is: does anyone know a 'better' way to filter a listview in wpf or should I filter the underlying ObservableCollection instead?

Britt

+1 Good question! What solution did you come up with? – Lunarian

It's possible the performance issue is on the grid binding/rendering side of things and not related to the actual filter logic... – Decoration

Answer 1

Pay close attention to your filter function. Make sure you aren't doing any unnecessary boxing/unboxing and you aren't doing extensive calculations in it. You should also pay attention to which kind of CollectionView you're using, some are faster than others. From Bea's post on sorting:

• A CollectionView is created if your source implements IEnumerable. If the source implements IEnumerable only, you will not be able to sort or group the collection (you can only filter it). Also, perf will not be the best if the source has a large number of items or if you perform dynamic operations such as insertions and deletions. If this is your scenario, you should consider having your source implement a stronger interface. ICollection is slightly better because it provides a Count property.

• ListCollectionView is the view type created when the source implements IList. Compared to IEnumerable and ICollection, IList performs much better for large or dynamic lists because it provides an indexer, allowing us quick random access. In addition, IList allows sorting, grouping and filtering. But ideally your source collection derives from ObservableCollection, the mother of all collections in the eyes of data binding, since it provides several extra goodies such as property and collection change notifications.

• BindingListCollectionView is the type of view created by Avalon when the source collection implements IBindingList. This is the view we deal with in the ADO.NET scenario. It supports sorting and grouping, but not traditional filtering. Instead, it has an additional CustomFilter property that delegates filtering to the DataView (see my post on ADO.NET for more details).

You can kick the filtering to a different thread as @mihi said but I have used CollectionViews to run multiple filters concurrently on an ObservableCollection with 50,000 items on an object with ~60 variables (columns in a database table) without noticeable lag.

One thing I notice immediately in your filter function is the input is of type Object which likely means you're doing a type conversion within the function and may not need to. You also use Predicate which doesn't include a return type so that probably requires some type conversions or reflection within the CollectionView's filtering methods and might slow you down as well.

Eugene

Could you please add some sample code to show how you would move the filtering to a different thread? I can't seem to get this working on a different Thread. – Aaberg

@Bryan: I have a similar situation and I would like to improve my filter performance. You wrote in your answer that one should avoid using a Predicate<object> because it requires boxing/unboxing operations and does not include a return type. What exactly do you mean by that? It has a return type of bool, and it must be a Predicate<object> because the ListCollectionView's Filter property is defined like that. Is there any way to get around this and use another filter approach? Or what was your intention? Maybe I just misunderstood it... – Lunarian

@Lunarian I think I was just throwing out ideas at the time for things that could be causing a slowdown. 99 times out of 100 it's going to be something slow in the filtering logic that's causing the problem. – Eugene

Answer 2

Have you considered filtering in another thread or using the dispatcher?

WPF Threads: Build more responsive apps with the dispatcher gives you a nice overview of some of the threading options available to you.

Busk

I've considered this, but I can't access the 'ICollectionView' in a different thread than the UI thread? – Britt

No, but you can perform the filtering logic in another thread and modify the collection on the UI thread using Dispatcher.Invoke – Diazonium

Here's a good example of this solution: codeproject.com/Articles/32426/… – Jaclynjaco