After using cURL to log into a remote webpage, how can I then access another webpage and return it as a string?

Question

Okay, I'm pretty new to cURL. I've managed to log into a webpage using cURL and sending POST data, but once I'm logged in, I'd like it to load various webpages under the same SESSION and extract data as a string, so that I can check if the webpage contains a specific string.

How would I go about doing this?

For example:

It logs into http://example.com/login.php

But after logging on, I need it to visit http://example.com/site.php?page=news and return the contents of that page as a string.

Answer 1

It generally depends on how the remote site handles state management.

In most cases, this will be via a cookie, so you'll need to instruct curl to keep track of cookies.

Something like:

curl_setopt($curl, CURLOPT_COOKIEJAR, '/tmp/cookies');  //where to write cookies received from server
curl_setopt($curl, CURLOPT_COOKIEFILE, '/tmp/cookies'); //where to read cookies to send in requests.

ought to get you started there.

EDIT Blatant Copy/Paste Example (from here - found with about 20 seconds of googling "php curl login"). This looks to be about right:

<?php 
$url ="http://login.yahoo.com/config/login?.src=ym&amp;.intl=us&amp;.partner=&amp;.done=http%3A%2F%2Fmail.yahoo.com%2F"; 
$ch = curl_init();      
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);  
curl_setopt($ch, CURLOPT_URL, $url); 
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY); 
curl_setopt ($ch, CURLOPT_COOKIEJAR, '/temp/cookie.txt'); 
curl_setopt ($ch, CURLOPT_POSTFIELDS, "login=emailid&passwd=password&&submit=Sign In"); 
ob_start();      
curl_exec ($ch); 
ob_end_clean();  
curl_close ($ch); 
unset($ch); 
$ch = curl_init(); 
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1); 
curl_setopt($ch, CURLOPT_COOKIEFILE, "/temp/cookie.txt"); 
curl_setopt($ch, CURLOPT_URL,"http://us.f515.mail.yahoo.com/ym/login?"); 
$result = curl_exec ($ch); 

curl_close ($ch); 
echo $result;