Parallel.ForEach throws exception when processing extremely large sets of data

Question

My question centers on some Parallel.ForEach code that used to work without fail, and now that our database has grown to 5 times as large, it breaks almost regularly.

Parallel.ForEach<Stock_ListAllResult>( lbStockList.SelectedItems.Cast<Stock_ListAllResult>(), SelectedStock =>
{
    ComputeTipDown( SelectedStock.Symbol );
} );

The ComputeTipDown() method gets all daily stock tic data for the symbol, and iterates through each day, gets yesterday's data and does a few calculations and then inserts them into the database for each day.

We use this rarely to recalculate static data values when a formula changes.

The exception is this:

The database we are hitting has 16 gigs of ram and is a dual quad-core and nobody was using the system while I was recalculating. The machine running the application to regenerate the code is a laptop with 12 gigs of ram with a hyper-threaded octal-core. So there was no obvious resource contention.

This is my foray into using .NET 4 and parallel processing, so I am wondering if there is something I am missing. Any thoughts would be welcomed.

Respirator

any solution with full source code ? – Pigeonwing

Answer 1

This looks like you received an AggregateException, which is what Parallel.ForEach will raise if any of the loop body methods raise an exception.

If you debug this, you should be able to look at the InnerExceptions to see the actual exceptions thrown. It looks like ComputeTipDown raised an exception in one or more of your iterations, causing this to occur.

Legislator

Reed, thank you very much for the suggestion, I'll put a breakpoint in the catch for ComputeTipDown and will see what happens. – Respirator

@MikeMalter: You can also wrap the Parallel.ForEach in a try/catch, and catch AggregateException. This will give you the details at that point... – Legislator

I read that by clicking on the AggregateException link you provided in your answer. I just got another exception, and this time it was the "real" one. Max pool size was reached. Interesting. I upgraded the server this weekend to 2008 r2 and SQL Server to 2008 R2, and it was after that this started happing with regular frequency. – Respirator

@MikeMalter: Are you running this as an x64 process? In x86, the default max thread size is quite a bit smaller... – Legislator

@MikeMalter: Is "ComputeTipDown" using a database internally? If so, you're probably hitting the max db connection pool. If that's the problem, you may want to set ParallelOptions.MaxDegreeOfParalellism and use a different overload of ForEach: msdn.microsoft.com/en-us/library/… – Legislator