Foreach uses some "magic" to figure out what type to loop over. It doesn't require the collection to support IEnumerable of anything. This probably explains the difference.
The problem is that the compiler can't figure out the generic type parameter for Where. If you do this, it will work:
IEnumerable<DerivedRecord> ie = derivedCollection;
ie.Where(d => d.DerivedProperty == "");
No casts are used, but the set of available choices has been reduced down to one.
Or you can specify the type parameter explicitly:
derivedCollection.Where<DerivedRecord>(d => d.DerivedProperty == "");
I think to solve your problem you're going to have to stop BaseCollection from inheriting another version of IEnumerable<T>:
abstract class BaseCollection
{
private readonly Collection<BaseRecord> _realCollection = new Collection<BaseRecord>();
public void Add(BaseRecord rec)
{
_realCollection.Add(rec);
}
public IEnumerable<BaseRecord> Items
{
get { return _realCollection; }
}
}
There I'm still instantiating Collection<T> but as a separate object instead of a base class. Then forwarding things on to it to mimic its API as necessary.
The derived generic version needs to completely implement IEnumerable<T>.
class BaseCollection<TRecord> : BaseCollection, IEnumerable<TRecord>
where TRecord : BaseRecord,new()
{
public IEnumerator<TRecord> GetEnumerator()
{
return Items.Cast<TRecord>().GetEnumerator();
}
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
{
return GetEnumerator();
}
}
Then the rest of your sample code works fine without the workarounds I posted initially, because it only has one IEnumerable<T> implementation to work with.
Update, more details
Here's a really slimmed down version of the same situation, breaking all links with IEnumerable, declaring everything ourselves from scratch, so we can see what's happening.
public class R1 { }
public class R2 { }
public interface I<T> { }
public class C1<T> : I<T> { }
public class C2 : C1<R1>, I<R2> { }
class Program
{
public static I<T> M<T>(I<T> i) { return i; }
static void Main(string[] args)
{
var c2 = new C2();
var v = M(c2); // Compiler error - no definition for M
}
}
R1 and R2 are like the two record classes. In your version, R2 inherits R1 - but that's not relevant to this issue, so I've omitted it. I<T> is the generic interface, standing in for IEnumerable<T>. But my version has no methods! They also aren't relevant to this issue.
Then I've collapsed your collection class inheritance system down to just two layers. The base class C1 implements I<T>, and then a derived class C2 chooses to ask C2 to implement I<R1> and then also implements I<R2> directly. The number of layers makes no difference either. Nor do type constraints declared with where, so they are also omitted here.
The upshot is that C2 has two implementations of I<T>: I<R1> and I<R2>. One it inherits from C1, the other it adds itself.
Finally I've got a method M, which stands in for Where in Linq. It doesn't need to be an extension method to show the issue, so I've made it an ordinary static method for clarity.
So when we come to call our method M, the compiler has to figure out what T is. It does this by looking at what we've passed as the only parameter to the method, which has to support I<T>. Unfortunately, we're passing something that supports I<R1> and I<R2>, so how can the type inference process make a choice between them? It can't.
As my interface has no methods, clearly putting new in front of a method isn't going to help me, and that's why it doesn't help you. The issue is not deciding which method in an interface to call, but whether to treat the argument to M as I<R1> or I<R2>.
Why doesn't the compiler report this as a type inference problem? According to the C# 3.0 spec, it just doesn't - type inference runs first, to produce a set of available overloads, and then overload resolution runs to pick the best choice. If type inference cannot decide between two possible expansions of a generic method, it eliminates both, so overload resolution never even sees them, so the error message says there isn't any applicable method called M.
(But if you have Resharper, it has its own compiler that it uses to give more detailed errors in the IDE, and in this case it says specifically: "The type arguments for method M cannot be inferred from the usage".)
Now, why is foreach different? Because it's not even type safe! It dates back to before generics were added. It doesn't even look at interfaces. It just looks for a public method called GetEnumerator in whatever type it loops through. For example:
public class C
{
public IEnumerator GetEnumerator() { return null; }
}
That's a perfectly good collection as far as the compiler is concerned! (Of course it will blow up at runtime because it returns a null IEnumerator.) But note the absence of IEnumerable or any generics. This means that foreach does an implicit cast.
So to relate this to your code, you have a "down cast" from BaseRecord to DerivedRecord, which you implement with the Cast operator from Linq. Well, foreach does that for you anyway. In my example above, C is effectively a collection of items of type object. And yet I can write:
foreach (string item in new C())
{
Console.WriteLine(item.Length);
}
The compiler happily inserts a silent cast from object to string. Those items could be anything at all... Yuck!
This is why the advent of var is great - always use var to declare your foreach loop variable, and that way the compiler will not insert a cast. It will make the variable be of the most specific type it can infer from the collection at compile time.