How to handle redirects while parsing HTML? - Python

Question

Hi folks,

I'm trying to submit a few forms through a Python script, I'm using the mechanized library.

This is so I can implement a temporary API.

The problem is that before after submission a blank page is returned informing that the request is being processed, after a few seconds the page is redirected to the final page.

---

I understand if it might sound a bit generic, but I'm not sure what is going on. :)

Any ideas?

Answer 1

Traditionally When you get a redirect, the status code of the response is 302, and there's a location header that instructs the browser where to go next. Other techniques(that are lame) would be to put a meta refresh tag in the head of the document.

<meta http-equiv="refresh" content="2;url=http://nextlocation.com"&gt;

And I suppose there's any number of ways to do it with javascript(also lame)

Answer 2

If it uses meta tags then you need to parse the HTML manually. Otherwise mechanize will handle the redirect automatically.