Application freezing while creating visual container for DataGrid rows

Question

I'm currently working on a problem that has me on it's toes for the last weeks or so.

This is not a normal freezing problem solveable by multi-threading, because I'm doing this already for the business logic.

I have a View with a DataGrid, the DataContext is a ViewModel supplying an ObservableCollection<T> to a CollectionViewSource named iGroupingJournal.

<DataGrid Grid.Row="1" ItemsSource="{Binding Source={StaticResource iGroupedJournal}}">
    <DataGrid.Columns>
        <DataGridTextColumn Binding="{Binding Path=DateTime}" Header="DateTime" />
        [...]
    </DataGrid.Columns>
</DataGrid>

Usually, about 50 rows having 7 columns are displayed, which causes the application to freeze while these rows are generated.

I already ruled out the following as causes for this:

1.1. complex business class producing overhead -> tested with a data holding class, produces no increase in speed

1.2. expensive database query gets executed in the gui thread due to getting the data via LINQ -> data gets fetched in the workerthread by caling ToList and mapping the data to the aforementioned holding classes

1.3. collection grouping increases overhead -> removed the grouping header in the CollectionViewSource.

The following was tested or thought about:

2.1. Tested UI Virtualisation by removing the grouping instruction, didn't have any effect, also grouping is a desireable feature

2.2. Data Virtualisation - won't have an effect because 50 rows have to be created as that's what usually shown

Conclusion

I did all I could to get the work out of the GUI thread, and it's clearly visible by a LoadingIndicator that the moment the workerthread sends NotifyCollectionChanged to the GUI thread, the whole application freezes. Did you encounter this problem before? How did you solve it? Or am I'm really out of luck?