Filtering IEnumerable Pattern

Question

Consider the followign simple code pattern:

foreach(Item item in itemList)
{
   if(item.Foo)
   {
      DoStuff(item);
   }
}

If I want to parallelize it using Parallel Extensions(PE) I might simply replace the for loop construct as follows:

Parallel.ForEach(itemList, delegate(Item item)
{
   if(item.Foo)
   {
      DoStuff(item);
   }
});

However, PE will perform unnecessary work assigning work to threads for those items where Foo turned out to be false. Thus I was thinking an intermediate wrapper/filtering IEnumerable might be a reasonable approach here. Do you agree? If so what is the simplest way of achieving this? (BTW I'm currently using C#2, so I'd be grateful for at least one example that doesn't use lambda expressions etc.

Answer 1

If I was to implement this, I would simply filter the list, before calling the foreach.

var actionableResults = from x in ItemList WHERE x.Foo select x;

This will filter the list to get the items that can be acted upon.

NOTE: this might be a pre-mature optimization, and could not make a major difference in your performance.

Answer 2

That all factors down to this single line:

Parallel.ForEach(itemList.Where(i => i.Foo), DoStuff);

But reading a comment to another post I now see you're in .Net 2.0 yet, so some of this may be a bit tricky to sneak past the compiler.

For .Net 2.0, I think you can do it like this (I'm a little unclear that passing the method names as delegates will still just work, but I think it will):

public IEnumerable<T> Where(IEnumerable<T> source, Predicate<T> predicate)
{
   foreach(T item in source)
       if (predicate(item))
          yield return item;
}

public bool HasFoo(Item item) { return item.Foo; }

Parallel.ForEach(Where(itemList, HasFoo), DoStuff);

Answer 3

I'm not sure how the partitioning in PE for .NET 2 works, so it's difficult to say there. If each element is being pushed into a separate work item (which would be a fairly poor partitioning strategy), then filtering in advance would make quite a bit of sense.

If, however, item.Foo happened to be at all expensive (I wouldn't expect this, given that it's a property, but it's always possible), allowing it to be parallelized could be advantageous.

In addition, in .NET 4, the partitioning strategy used by the TPL will handle this fairly well. It was specifically designed to handle situations with varying levels of work. It does partitioning in "chunks", so one item does not get sent to one thread, but rather a thread gets assigned a set of items, which it processes in bulk. Depending on the frequency of item.Foo being false, paralellizing (using TPL) would quite possibly be faster than filtering in advance.