+1  A: 

Was there a reason that the answer with this link was deleted?

Sam Holder
A: 

QuickGraph is really good general purpose graph library. It has some implementations for walking Markov Chains. I recommend checking out the source code.

Eric Hauser