using Live HTTP Headers

Question

On a website, I enter some parameters in a form, click on search and then get a page with a message "retrieving your results". After the search is complete, I get another page with my results displayed.

I am trying to recreate this programatically and I used Live HTTP Headers to get a peek of what is going on behind i.e the url, form variables,etc. However, I'm only getting information of what goes on up to the page which shows "retrieving your results". Live HTTP Header is not giving me information up to the page which contains the final results.

What can I do to get this final bit of information (i.e the url, form variables, etc)

Answer 1

"Web Developer" does this:

https://addons.mozilla.org/en-US/firefox/addon/60

Answer 2

I use Charles HTTP Proxy for all my HTTP troubleshooting needs. It has a ton of options and works with any browser.

Answer 3

@Mark Harrison

I have webdeveloper installed. Initially, I used it to turn off meta-redirects and referrers to get a clearer picture of the http interaction. But when i do this, the website does not work (i.e it is not able to complete the process of retrieving my search results) so i turned it back on.

I'm wondering if anyone has had to capture http information for a site that has a processing page in between the user input page and the results page

Answer 4

That sounds weird? I'm pretty sure that LiveHttpHeaders should show this. Can you double check that you aren't missing something? Otherwise try with Firebug. It has a tab for "network", which shows all requests made.

Answer 5

I'm using Fiddler2, which is a free (as in beer), highly configurable proxy; works with all browsers, allows header inspection/editing/automodification on request/response.

Disclaimer: I'm in no way affiliated with Fiddler, just a (very happy) user.

Answer 6

I for such problems always fire-on an Ethereal or similar network spying tool, to see exactly, what is going on.

Answer 7

The document is creating a browser component called XMLHTTPRequest , on submit event the object method send() is called, during the waiting time for server response an html element is replaced with a "Waiting message" on succesfull response a callback is called with the new html elements and then inserted in the selected html element. (That's called ajax).

If you want to follow that process you can use Firefox Live HTTP Headers Extension , or Wireshark to view full HTTP headers and actions (get/post/).