I'm looking for a PHP library that allows me to scrap webpages and takes care about all the cookies and prefilling the forms with the default values, that's what annoys me the most.
I'm tired of having to match every single input element with xpath and I would love if something better existed. I've come across phpQuery but the manual isn't much clear and I can't find out how to make POST requests.
Can someone help me? Thanks.
@Jonathan Fingland:
In the example provided by the manual for browserGet() we have:
require_once('phpQuery/phpQuery.php');
phpQuery::browserGet('http://google.com/', 'success1');
function success1($browser)
{
$browser->WebBrowser('success2')
->find('input[name=q]')->val('search phrase')
->parents('form')
->submit();
}
function success2($browser)
{
echo $browser;
}
I suppose all the other fields are scrapped and send back in the GET request, I want to do the same with the phpQuery::browserPost() method but I don't know how to do it. The form I'm trying to scrape has a input token and I would love if phpQuery could be smart enough to scrape the token and just let me change the other fields (in this case username and password), submiting via POST everything.
PS: Rest assured, this is not going to be used for spamming.