Why doesn't deferred execution cache iterative values?

Question

Take the code below, adapted from this question:

//Borrowed from another question because its a simpler example of what happened to me.
IEnumerable<char> query = "Not what you might expect";
foreach(char vowel in "aeiou")
{
     query = query.Where(c => c != vowel);
}
foreach (char Output in query)
{
    System.Out.WriteLine(Output);
}

This only removes the 'u' from the query char collection. The core issue has something to do with the fact that the 'c' variable in the Where clause isn't evaluated until the second foreach. My question is:

1) Why would the delegate generated by the first foreach not capture each value of c as it is built up? Is there some situation I'm unaware of where that is not the desired behavior?

2) If its not capturing the value of c, how is that value even still in scope in the second foreach when the query is actually run? It would seem to me that if its not storing the values of the variables being passed in, then trying to resolve the statement for the second foreach would fail because the the variable c is clearly out of scope.

I don't understand how it is that 'use the last value we saw on this variable back when it was in scope' was a good design decision for this circumstance, and was hoping someone could shed some light on the subject.

Answer 1

It is capturing vowel; try:

foreach(char vowel in "aeiou") {
     char tmp = vowel;
     query = query.Where(c => c != tmp);
}

Jon has a list of related posts here.

As for why... foreach is defined (in ECMA 334v4 §15.8.4) as:

A foreach statement of the form `foreach (V v in x) embedded-statement` is then expanded to:

{
  E e = ((C)(x)).GetEnumerator();
  try {
    V v;
    while (e.MoveNext()) {
      v = (V)(T)e.Current;
      embedded-statement
    }
  }
  finally {
    … // Dispose e
  }
}

Note the V is outside the while - this means it is captured once only for the foreach. I gather that the C# team have, at times, doubted the wisdom of this change (it was inside the while in the C# 1.2 spec, which would have avoided the problem completely). Maybe it'll change back eventually, but for now it is being true to the specification.

Answer 2

As Marc says, it's capturing vowel as a variable, rather than the value of vowel in each case. This is almost always undesirable, but it is the specified behaviour (i.e. the compiler is following the language spec). There's a single vowel variable, rather than a "new variable instance" per iteration.

Note that there's no expression tree involved here - you're using IEnumerable<T> so it's just converting the lambda expression into a delegate.

See section 7.14.4 of the C# 3 spec for more information.

Answer 3

I think what you really want to do here is something more like this:

IEnumerable<char> query = "Not what you might expect";
query = query.Except("aeiou");

foreach (char Output in query)
{
    System.Out.WriteLine(Output);
}