Complex retrieval of a web document

Question

I need to retrieve a document from a website, and parse it. Problem is that:

1. The site uses both http and https protocol
2. You need to log in the site (I have a regular account)
3. From the login page, there are at least 2 redirect just to log in yourself

I managed an HTTPS connection and posted my login and pass, but I'm having troubles with cookie management and the redirect....

Answer 1

Using a library like HtmlUnit would probably help.

Answer 2

commons-httpclient would help.