what causes an f# sequence to re-enumerate?

Question

i have a parser that is effectively a set of recursive functions operating on a sequence of lexer tokens. The problem i'm running into is that the sequence seems to restart from the beginning on recursive function calls. Given the following skeleton definition for function Parse

    let restricted = Seq.take_while token_search tokens
    let compiled_nodes = Seq.fold (fun list (next: Lexer.Token) -> list @ parse_token this restricted next) [] restricted

the function parse_token may result in a call into Parse. However, when that happens, the parameter tokens ends up positioned at the beginning of the sequence. Any ideas on how to keep the sequence positioned where it needs to be?

tia

Answer 1

I think you may need to post a slightly bigger snippet, as I am not quite following you.

That said, a sequence (IEnumerable) is just that - a sequence, and each time you for (foreach) or Seq.Whatever over it, it will 're-iterate' the sequence. I am unclear what you want to do, and what you expect to happen, but for a parse, representing 'tokens' as a sequence may be 'wrong', as you typically partition tokens into a consumed/committed region and a lookahead region.

Note also that you typically do not want 'iterating over a sequence' to have side-effects.