WPF's ICollectionView.filter with large sets of data

Question

Hello,

I'm working on an wpf app which contains a listview with quite a lot of data (10 000 to 100 000) rows. The user can apply all sorts of filters to this listview making the filter logic quite advanced (and slow). For now, the relevant part of my code looks like this:

ICollectionView view = CollectionViewSource.GetDefaultView(hugeList.ItemsSource);
view.Filter = new Predicate<object>(FilterCallback);

private bool FilterCallback(object item)
{
  //Filter logic
}

But this runs in the UI thread and blocks the entire application when filtering which gives a very poor user experience. So my question to you all is: does anyone know a 'better' way to filter a listview in wpf or should I filter the underlying ObservableCollection instead?

Answer 1

Have you considered filtering in another thread or using the dispatcher?

WPF Threads: Build more responsive apps with the dispatcher gives you a nice overview of some of the threading options available to you.

Answer 2

Pay close attention to your filter function. Make sure you aren't doing any unnecessary boxing/unboxing and you aren't doing extensive calculations in it. You should also pay attention to which kind of CollectionView you're using, some are faster than others. From Bea's post on sorting:

• A CollectionView is created if your source implements IEnumerable. If the source implements IEnumerable only, you will not be able to sort or group the collection (you can only filter it). Also, perf will not be the best if the source has a large number of items or if you perform dynamic operations such as insertions and deletions. If this is your scenario, you should consider having your source implement a stronger interface. ICollection is slightly better because it provides a Count property.

• ListCollectionView is the view type created when the source implements IList. Compared to IEnumerable and ICollection, IList performs much better for large or dynamic lists because it provides an indexer, allowing us quick random access. In addition, IList allows sorting, grouping and filtering. But ideally your source collection derives from ObservableCollection, the mother of all collections in the eyes of data binding, since it provides several extra goodies such as property and collection change notifications.

• BindingListCollectionView is the type of view created by Avalon when the source collection implements IBindingList. This is the view we deal with in the ADO.NET scenario. It supports sorting and grouping, but not traditional filtering. Instead, it has an additional CustomFilter property that delegates filtering to the DataView (see my post on ADO.NET for more details).

You can kick the filtering to a different thread as @mihi said but I have used CollectionViews to run multiple filters concurrently on an ObservableCollection with 50,000 items on an object with ~60 variables (columns in a database table) without noticeable lag.

One thing I notice immediately in your filter function is the input is of type Object which likely means you're doing a type conversion within the function and may not need to. You also use Predicate which doesn't include a return type so that probably requires some type conversions or reflection within the CollectionView's filtering methods and might slow you down as well.