views:

44

answers:

0

Possible Duplicate:
LINQ Partition List into Lists of 8 members.

I've got an IEnumerable<T> and I'd like to transform it into an IEnumerable<List<T>> where each List is a batch of items in the same order as the original enumerator. Each batch should be batchSize items in length, except for the last batch which should contain leftover items, so it may be smaller than batchSize.

The underlying store is not a database or other store where I can push the batching into a lower level of the system. Instead, I need to accumulate the batches myself, because it's a stream of objects that I only can fetch one at a time.

The sequence is very long and fetching takes a while, so I can't package the entire enumerable into a List<T> or array ahead of time. Instead, I'd like to fetch batchSize results and then immediately start processing the first batch.

I may decide to later make the fetching asynchronous to the processing (so that processing batch 1 and fetching batch 2 can happen in parallel) but for now I'm just looking for a single-threaded solution.

Any suggestions for how to do this batched, streaming enumeration efficiently and elegantly? Ideally I'd like to package it up into an extension method I can re-use, e.g.

public static IEnumerable<T> Batch<T>(this IEnumerable<T> source, int count);