How to parse a rendered web page containing javascript.

Question

How can one extract data from a rendered web page? In which java script would update the data with time. Is it possible to write user script which can access varibles from webpage java script? Please suggest possible way to achieve this.

Answer 1

according to Turing's Halting Problem Theorem, you can't.

That's what we mean when we say that JavaScript is a Turing complete language. The only way is to execute the JavaScript and let it render the page.

Answer 2

it depends on your programming language.

In C# you could use a webbrowser control, then use the Webbrowser.Document property to get a HTMLDocument object to get the current markup. To invoke javascript function in the document, use the ObjectForScripting property of the Browser control.