Ensure that the underlying collection implements from INotifyPropertyChanged interface, and OnPropertyChanged is called when the row is deleted.
ObservableCollection is a colelction that implements INotifyPropertyChanged so consider using that as an underlying collection object.
MSDN Remarks on CollectionViewSource
CollectionViewSource is a proxy for a
CollectionView class, or a class
derived from CollectionView.
CollectionViewSource enables XAML code
to set the commonly used
CollectionView properties, passing
these settings to the underlying view.
CollectionViewSource has a View
property that holds the actual view
and a Source property that holds the
source collection.
You can think of a collection view as
the layer on top of the binding source
collection that allows you to navigate
and display the collection based on
sort, filter, and group queries, all
without having to manipulate the
underlying source collection itself.
If the source collection implements
the INotifyCollectionChanged
interface, the changes raised by the
CollectionChanged event are propagated
to the views.
Because views do not change the
underlying source collections, each
source collection can have multiple
views associated with it. For example,
you may have a collection of Task
objects. With the use of views, you
can display that same data in
different ways. For example, on the
left side of your page you may want to
show tasks sorted by priority, and on
the right side, grouped by area.
For more information, see the Binding
to Collections section in the Data
Binding Overview.