Simulating a cookie-enabled browser in PHP

Question

How can I open a web-page and receive its cookies using PHP?

The motivation: I am trying to use feed43 to create an RSS feed from the non-RSS-enabled HighLearn website (remote learning website). I found the web-page that contains the feed contents I need to parse, however, it requires to login first. Luckily, logging in can be done via a GET request so it's as easy as fopen()ing "http://highlearn.website/login_page.asp?userID=foo&password=bar" for example. But I still need to get the cookies generated when I logged in, pass the cookies to the real client (using setcookie() maybe?) and then redirect.

Answer 1

Unfortunately, this is not possible unless the websites are on the same domain. Cookies are only valid on the domain they originated from. Also, subdomains count as different domains. Otherwise, (keep in mind I haven't used much php) you could pull the headers out of the response, and copy the cookie out to the client.

Answer 2

For a server-side HTTP client you should use the cURL module. It will allow you to persist cookies across multiple requests. It also does some other neat things like bundling requests (curl_multi) and transparently handling redirects.

When it comes to returning a session to your user, I don't think this is possible. You would need to be able to overwrite the cookies of other domains. This would cause massive security issues, so no browser would implement it.

Answer 3

I've used the Scriptable Browser component from Simpletest for this kind of screen scraping before. It does a pretty good job of simulating a browser.

You don't need to pass the session on to the real client (Even though it may be possible, depending on the site's security level) - You can simply let your PHP-script be a proxy between the target site and your end-user.