views:

838

answers:

6

In python the yield keyword can be used in both push and pull contexts, I know how to do the pull context in c# but how would I achieve the push. I post the code I am trying to replicate in c# from python:

def coroutine(func):
  def start(*args,**kwargs):
    cr = func(*args,**kwargs)
    cr.next()
    return cr
  return start

@coroutine
def grep(pattern):
  print "Looking for %s" % pattern
  try:
    while True:
      line = (yield)
      if pattern in line:
        print line,
  except GeneratorExit:
    print "Going away. Goodbye"
+3  A: 

Maybe this will help.

http://blogs.msdn.com/ericlippert/archive/2009/07/23/iterator-blocks-part-five-push-vs-pull.aspx

NickLarsen
I know it can be done, saving the state as an object and calling its methods, but what about a straight out and out co-routine.
WeNeedAnswers
Yes, I seen that, and I have looked at the IObservable stuff thats beeing introduced, hang on there, I just had one of those "click moments" :)
WeNeedAnswers
+2  A: 

thanks @NickLarsen, you helped me remember the new stuff that MS have introduced, the IObservable interface.

link http://msdn.microsoft.com/en-us/library/dd783449(VS.100).aspx

WeNeedAnswers
+3  A: 

C# does not have general co-routines. A general co-routine is where the co-routine has its own stack, i.e. it can invoke other methods and those methods can "yield" values. Implementation of general co-routines requires making some smart things with stacks, possibly up to and including allocating stack frames (the hidden structures which contain local variables) on the heap. This can be done, some languages do that (e.g. Scheme), but it is somewhat tricky to do it right. Also, many programmers find the feature difficult to understand.

General co-routines can be emulated with threads. Each thread has its own stack. In a co-routine setup, both threads (the initial caller, and the thread for the co-routine) will alternate control, they will never actually run simultaneously. The "yield" mechanism is then an exchange between the two threads, and as such it is expensive (synchronization, a roundtrip through the OS kernel and scheduler...). Also, there is much room for memory leaks (the co-routine must be explicitly "stopped", otherwise the waiting thread will stick forever). Thus, this is rarely done.

C# provides a bastardized-down co-routine feature called iterators. The C# compiler automatically converts the iterator code into a specific state class, with local variables becoming class fields. Yielding is then, at the VM level, a plain return. Such a thing is doable as long as the "yield" is performed from the iterator code itself, not from a method which the iterator code invokes. C# iterators already cover many use cases and the C# designers were unwilling to go further down the road to continuations. Some sarcastic people are keen to state that implementing full-featured continuations would have prevented C# from being as efficient as its arch-enemy Java (efficient continuations are feasible, but this requires quite some work with the GC and the JIT compiler).

Thomas Pornin
nice answer. I think I found what I was trying to do with C#, IObservable. That stuff about the stack, does Python under the covers do that, because I don't think it does. It uses vanilla C. I don't know for certain though. I am trying to avoid all use of threads, hence why I was interested in the co-routine root. I find that if I am in full control of my programming, and I got the tools to avoid threads, why not try. The coroutine stuff should work a treat all be it wrapped up in IObservable fakeness :)
WeNeedAnswers
Java efficient...:) sorry that made me smile. I remember doing a solution using JBoss. the thing wouldn't release objects, we told our end customer (T.U.I. --- Thomas Cook) that he would need to switch it on and off again once in a while. :)
WeNeedAnswers
If you do not like Java, efficiency-wise, then chances are that you will not be very happy with C# either. They are quite similar in that respect. Actually they are quite similar in every view.
Thomas Pornin
very true, I have worked with both and they do feel similar. I get better memory foot prints with .Net than Java. I think it has something to do with Deterministic disposal. I wouldn't use either given the choice. But I must use what the industry demands. I would use F# or python.
WeNeedAnswers
+7  A: 

If what you want is an "observable collection" -- that is, a collection which pushes results at you rather than letting the consumer pull them -- then you probably want to look into the Reactive Framework extensions. Here's an article on it:

http://www.infoq.com/news/2009/07/Reactive-Framework-LINQ-Events

Now, as you note, you can build both "push" and "pull" style iterators easily if you have coroutines available. (Or, as Thomas points out, you can build them with continuations as well.) In the current version of C# we do not have true coroutines (or continuations). However, we are very concerned about the pain users feel around asynchronous programming.

Implementing fiber-based coroutines as a first-class language feature is one technique that could possibly be used to make asynchronous programming easier, but that is just one possible idea of many that we are at present researching. If you have a really solid awesome scenario where coroutines do a better job than anything else -- including the reactive framework -- then I'd love to hear more about it. The more realistic data we have about what real problems people are facing in asynchronous programming, the more likely we are to come up with a good solution. Thanks!

Eric Lippert
One such scenario is Event Driven Simulation (Simula style). I have a project using Win32 fibers that I still can't port to .NET
Henk Holterman
I think asynchronous programming in C# is great. I like the Erlang styled MailboxProcessor in F# though. As for co-routines, I find them very interesting. I know that they are old technology and were swept away when threads were introduced, but I like the idea that I know exactly when a routine will execute and when it will pass control back. The main problem I find with threads is the lack of control and the general heaviness of creating them. They do however provide a nice level of thought abstraction, that require very little effort to implement :)
WeNeedAnswers
A: 

I would love to see a fiber-based API for .Net.

I attempted to use the native fiber API in C# through p/invoke a while back, but because the runtime's exception handling (incorrectly) makes thread-based assumptions, things broke (badly) when exceptions happened.

One "killer app" for a fiber-based coroutine API is game programming; certain types of AI require a "lightweight" thread that you can time-slice at will. For example, game behavior trees require the ability to "pulse" the decision code every frame, allowing the AI code to cooperatively yield back to the caller when the decision slice is up. This is possible to implement with hard threads, but much, much more complicated.

So while true fiber use-cases are not mainstream, they definitely exist, and a small niche of us .Net coders would cheer mightily if the existing bugs in the fiber subsystem were worked out.

A: 

Actually .NET does not make "incorrect assumptions" about thread affinity, in fact it totally decouples the notion of a .NET level thread from the OS level thread.

What you have to do is associate a logical .NET thread state with your fiber ( for that you need the CLR Hosting API's but you don'T need to write a host yourself you can use those needed from your own application directly ) and everything, lock tracking, exception handling works normally again.

An example can be found here: http://msdn.microsoft.com/en-us/magazine/cc164086.aspx

Btw Mono 2.6 contains low level Coroutine support and can be used to implement all higher level primitives easily.

Gluber