Programmatically Submit form and loop through paging (C#.NET)

Question

I need to write a custom web-scraper to mine some data. ?I know how to submit a form using HttpWebRequest class Post method. My challenge is to loop through the resulting pages and retrieve the records from each page.

Does anyone have a code sample or article to point to? Thanks

Answer 1

I recommend WatiN