file_get_contents from url that is only accessible after log-in to website

Question

I would like to make a php script that can capture a page from a website. Think *file_get_contents($url)*.

However, this website requires that you fill in a username/password log-in form before you can access any page. I imagine that once logged-in, the website sends your browser an authentication cookie and with every consequent browser request, the session info is passed back to the website to authenticate access.

I want to know how i can simulate this behavior of the browser with a php script in order to gain access and capture a page from this website.

More specifically, my questions are:

1. How do I send a request that contains my log-in details so that the website replies with the session information/cookie
2. How do i read the session information/cookie
3. How do i pass back this session information with every consequent request (*file_get_contents*, curl) to the website.

Thanks.

Answer 1

The http pecl extension can do it, also PEAR::HTTP_Client, Snoopy and many many other libraries/classes. If you (for whatever reason) you want to achieved this with file_get_contents, you can use the stream context options for the http wrapper to set POST and cookie parameters and stream_get_meta_data to read the response headers including the cookies.

Answer 2

Curl is pretty well suited to do it. You don't need to do anything special other than set the CURLOPT_COOKIEJAR and CURLOPT_COOKIEFILE options. Once you've logged in by passing the form fields from the site the cookie will be saved and Curl will use that same cookie for subsequent requests automatically as the example below illustrates.

Note that the function below saves the cookies to 'cookies/cookie.txt' so make sure that directory/file exists and can be written to.

$loginUrl = 'http://example.com/login'; //action from the login form
$loginFields = array('username'=>'user', 'password'=>'pass'); //login form field names and values
$remotePageUrl = 'http://example.com/remotepage.html'; //url of the page you want to save  

$login = getUrl($loginUrl, 'post', $loginFields); //login to the site

$remotePage = getUrl($remotePageUrl); //get the remote page

function getUrl($url, $method='', $vars='') {
    $ch = curl_init();
    if ($method == 'post') {
        curl_setopt($ch, CURLOPT_POST, 1);
        curl_setopt($ch, CURLOPT_POSTFIELDS, $vars);
    }
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
    curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookies/cookies.txt');
    curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookies/cookies.txt');
    $buffer = curl_exec($ch);
    curl_close($ch);
    return $buffer;
}