HTTP Request, loading javascript DOM manipulations that have been made to the HTML

Question

I'm currently using cURL to do HTTP requests, and it works fine. However I need to get the javascript code and execute it in the context of the HTML, making it manipulate the DOM exactly as if it were a web-browser.

The first thing that came to mind was to use firefox, there's a command-line interface so I thought it would be easy (maybe with some add-on) to programmatically do an HTTP request, let it natively run the javascript and manipulate the DOM, and get the generated HTML after the manipulation.

However this is harder than I expected, given also the fact that there's going to be problems fetching the data asynchronously.

Maybe someone has done this already and could give me some tips on what would be the best solution.

Answer 1

Hi, Luca,

You could probably use Selenium remote control to achieve this.

Cheers, V.

Answer 2

This is what you want to use for something like this:

http://code.google.com/p/envjs/

Answer 3

I would recommend Watir