Sorting and Filtering a collection in a Paged ListBox

Question

I have a database with 10,000 items, to which you can add and remove while the app is running.

I have a ListBox that displays at most 100 items, and supports paging.

You can filter and sort on the 10,000 items, which needs to be immediately reflected in the listbox.

I have a button that randomly selects an item as long as it passes the filters.

What is the best set of collections/views to use for this kind of operation?

So far, my first step will be to create an ObservableCollection of ALL items in the database which we will call MainOC.

Then create a List of all items that match the filter by parsing MainOC which we will call FilteredList.

Then create a ListCollectionView based on the above List that holds the first 100 items.

CONS:

• You have to recreate the ListCollectionView every time a sort operation is applied.
• You have to recreate the ListCollectionView every time you page.
• You have to recreate the ListCollectionView every time a filter is changed.
• You have to recreate the ListCollectionView every time an item is added or removed to MainOC.

Is there a better approach that I am missing?

For example, I see that you can apply filters to a ListCollectionView. Should I populate my ListCollectionView with all 10,000 items? But then how can I limit how many items my ListBox is displaying?

Should I be doing my filtering and sorting directly against the database? I could build FilteredList directly off the database and create my ListCollectionView based off that, but this still has all the cons listed above.

Looking for any input you can provide!

Mcquiston

Answer 1

This is a problem which is easily solved using DynamicData . Dynamic data is based on rx so if you are not familiar with the wonderful Rx I suggest you start learning it. There is quite a bit of a learning curve but but the rewards are huge.

Anyway back to my answer, the starting point of dynamic data is to get some data into a cache which is constructed with a key as follows

var myCache = new SourceCache<MyObject, MyId>(myobject=>myobject.Id)

Obviously being a cache there are methods to add, update and remove so I will not show those here.

Dynamic data provides a load of extensions and some controllers to dynamically interrogate the data. For paging we need a few elements to solve this problem

//this is an extension of observable collection optimised for dynamic data
var collection = new ObservableCollectionExtended<MyObject>();
//these controllers enable dynamically changing filter, sort and page
var pageController = new PageController();  
var filterController = new FilterController<T>(); 
var sortController = new SortController<T>(); 

Create a stream of data using these controllers and bind the result to the collection like this.

var mySubscription = myCache.Connect()
    .Filter(filterController)
    .Sort(sortController)
    .Page(pageController)
    .ObserveOnDispatcher() //ensure we are on the UI thread
    .Bind(collection)
    .Subscribe() //nothing happens until we subscribe.  

At any time you can change the parameters of the controllers to filter, sort, page and bind the data like follows

//to change page
pageController.Change(new PageRequest(1,100));
//to change filter 
filterController.Change(myobject=> //return a predicate);
//to change sort
sortController .Change( //return an IComparable<>);

And as if by magic the observable collection will self-maintain when any of the controller parameters change or when any of the data changes.

The only thing you now have to consider is the code you need for loading the database data into the cache.

In the near future I will create a working example of this functionality.

For more info on dynamic data see

Dynamic data on Github

Wpf demo app

Narcotism

Install Rx-Xaml from nuget and include namespace System.Reactive.Linq – Narcotism

tried to setup a super simple example. gist.github.com/julesx/acd0e1c03f9d90bd6094. Nothing shows up when I add to the cache, only when I add directly to the OC. Am i missing something basic? – Mcquiston

I will take a look over the weekend. Cannot look now as the office proxy blocks gists! – Narcotism

@RolandPheasant thank you for your work. I've bookmarked your library in WPF Room – Fiberboard

@RolandPheasant realized the filter controller is filtering out all the items by default! Thanks again! – Mcquiston

Are PageController, FilterController<T> and SortController<T> obsolete? I cannot find them in current NuGet package DynamicData 6.1.4 – Coryza

Is my understanding correct, that all of the 10k items need to be loaded into SourceCache? What if I had so many item that do not fit in RAM? Or what if the underlying database rows are not static but changed by another process? Is there a way of querying page by page from the database? – Coryza