Why don't the Linq extension methods sit on IEnumerator rather than IEnumerable?

Question

There are lots of Linq algorithms that only need to do one pass through the input e.g. Select.

Yet all the Linq extension methods sit on IEnumerable rather than IEnumerator

    var e = new[] { 1, 2, 3, 4, 5 }.GetEnumerator(); 
    e.Select(x => x * x); // Doesn't work 

This means you can't use Linq in any situation where you are reading from an "already opened" stream.

This scenario is happening a lot for a project I am currently working on - I want to return an IEnumerator whose IDispose method will close the stream, and have all the downstream Linq code operate on this.

In short, I have an "already opened" stream of results which I can convert into an appropriately disposable IEnumerator - but unfortunately all of the downstream code requires an IEnumerable rather than an IEnumerator, even though it's only going to do one "pass".

i.e. I'm wanting to "implement" this return type on a variety of different sources (CSV files, IDataReaders, etc.):

class TabularStream 
{ 
    Column[] Columns; 
    IEnumerator<object[]> RowStream; 
}

In order to get the "Columns" I have to have already opened the CSV file, initiated the SQL query, or whatever. I can then return an "IEnumerator" whose Dispose method closes the resource - but all of the Linq operations require an IEnumerable.

The best workaround I know of is to implement an IEnumerable whose GetEnumerator() method returns the one-and-only IEnumerator and throws an error if something tries to do a GetEnumerator() call twice.

Does this all sound OK or is there a much better way for me to implement "TabularStream" in a way that's easy to use from Linq?

Answer 1

Using IEnumerator<T> directly is rarely a good idea, in my view.

For one thing, it encodes the fact that it's destructive - whereas LINQ queries can usually be run multiple times. They're meant to be side-effect-free, whereas the act of iterating over an IEnumerator<T> is naturally side-effecting.

It also makes it virtually impossible to perform some of the optimizations in LINQ to Objects, such as using the Count property if you're actually asking an ICollection<T> for its count.

As for your workaround: yes, a OneShotEnumerable would be a reasonable approach.