why iterator methods can't take either 'ref' or 'out' parameters?

Question

I tried this earlier today:

public interface IFoo
{
    IEnumerable<int> GetItems_A( ref int somethingElse );
    IEnumerable<int> GetItems_B( ref int somethingElse );
}


public class Bar : IFoo
{
    public IEnumerable<int> GetItems_A( ref int somethingElse )
    {
        // Ok...
    }

    public IEnumerable<int> GetItems_B( ref int somethingElse )
    {
        yield return 7; // CS1623: Iterators cannot have ref or out parameters            

    }
}

What's the rationale behind this? Thanks in advance.

Answer 1

C# iterators are state machines internally. Every time you yield return something, the place where you left off should be saved along with the state of local variables so that you could get back and continue from there.

To hold this state, C# compiler creates a class to hold local variables and the place it should continue from. It's not possible to have a ref or out value as a field in a class. Consequently, if you were allowed to declare a parameter as ref or out, there would be no way to keep the complete snapshot of the function at the time we had left off.

EDIT: Technically, not all methods that return IEnumerable<T> are considered iterators. Just those that use yield to produce a sequence directly are considered iterators. Therefore, while the splitting the iterator into two methods is a nice and common workaround, it doesn't contradict with what I just said. The outer method (that doesn't use yield directly) is not considered an iterator.

Answer 2

At a highish level, A ref variable can point to many locations including to value types that are on the stack. The time at which the iterator is initially created by calling the iterator method and when the ref variable would be assigned are two very different times. It is not possible to guarantee that the variable which originally was passed by reference is still around when the iterator actually executes. Hence it is not allowed (or verifiable)

Answer 3

If you want to return both an iterator and an int from your method, a workaround is this:

public class Bar : IFoo
{
    public IEnumerable<int> GetItems( ref int somethingElse )
    {
        somethingElse = 42;
        return GetItemsCore();
    }

    private IEnumerable<int> GetItemsCore();
    {
        yield return 7;
    }
}

You should note that none of the code inside an iterator method (i.e. basically a method that contains yield return or yield break) is executed until the MoveNext() method in the Enumerator is called. So if you were able to use out or ref in your iterator method, you would get surprising behavior like this:

// This will not compile:
public IEnumerable<int> GetItems( ref int somethingElse )
{
    somethingElse = 42;
    yield return 7;
}

// ...
int somethingElse = 0;
IEnumerable<int> items = GetItems( ref somethingElse );
// at this point somethingElse would still be 0
items.GetEnumerator().MoveNext();
// but now the assignment would be executed and somethingElse would be 42

This is a common pitfall, a related issue is this:

public IEnumerable<int> GetItems( object mayNotBeNull ){
  if( mayNotBeNull == null )
    throw new NullPointerException();
  yield return 7;
}

// ...
IEnumerable<int> items = GetItems( null ); // <- This does not throw
items.GetEnumerators().MoveNext();                    // <- But this does

So a good pattern is to separate iterator methods into two parts: one to execute immediately and one that contains the code that should be lazily executed.

public IEnumerable<int> GetItems( object mayNotBeNull ){
  if( mayNotBeNull == null )
    throw new NullPointerException();
  // other quick checks
  return GetItemsCore( mayNotBeNull );
}

private IEnumerable<int> GetItemsCore( object mayNotBeNull ){
  SlowRunningMethod();
  CallToDatabase();
  // etc
  yield return 7;
}    
// ...
IEnumerable<int> items = GetItems( null ); // <- Now this will throw

EDIT: If you really want the behavior where moving the iterator would modify the ref-parameter, you could do something like this:

public static IEnumerable<int> GetItems( Action<int> setter, Func<int> getter )
{
    setter(42);
    yield return 7;
}

//...

int local = 0;
IEnumerable<int> items = GetItems((x)=>{local = x;}, ()=>local);
Console.WriteLine(local); // 0
items.GetEnumerator().MoveNext();
Console.WriteLine(local); // 42