Parallel.ForEach throws exception when processing extremely large sets of data

Question

My question centers on some Parallel.ForEach code that used to work without fail, and now that our database has grown to 5 times as large, it breaks almost regularly.

Parallel.ForEach<Stock_ListAllResult>( lbStockList.SelectedItems.Cast<Stock_ListAllResult>(), SelectedStock =>
{
    ComputeTipDown( SelectedStock.Symbol );
} );

The ComputeTipDown() method gets all daily stock tic data for the symbol, and iterates through each day, gets yesterday's data and does a few calculations and then inserts them into the database for each day.

We use this rarely to recalculate static data values when a formula changes.

The exception is this:

The database we are hitting has 16 gigs of ram and is a dual quad-core and nobody was using the system while I was recalculating. The machine running the application to regenerate the code is a laptop with 12 gigs of ram with a hyper-threaded octal-core. So there was no obvious resource contention.

This is my foray into using .NET 4 and parallel processing, so I am wondering if there is something I am missing. Any thoughts would be welcomed.

Answer 1

This looks like you received an AggregateException, which is what Parallel.ForEach will raise if any of the loop body methods raise an exception.

If you debug this, you should be able to look at the InnerExceptions to see the actual exceptions thrown. It looks like ComputeTipDown raised an exception in one or more of your iterations, causing this to occur.