Bypass authenticated XML page to be parsed with PHP?

Question

Hello,

How can I parse this xml page http://evercore:[email protected]/www.sportsbook.com/trends/203.xml using PHP simplexml? I get an error saying it can't be loaded.

Thanks, S

Answer 1

You need to spoof the Agent to get it to accept requests.

Use this PHP code to get the result:

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://feeds.outgard.net/www.sportsbook.com/trends/203.xml');
curl_setopt($ch, CURLOPT_GET, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FAILONERROR, true);
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
curl_setopt($ch, CURLOPT_USERPWD, 'evercore:david10');
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT     5.0)"); 

$data = curl_exec($ch);
echo $data;

// Process data with simple XML

curl_close($ch); 

Answer 2

Was toying around with this until @vassilis pointed out user-agent sniffing is applied, see it as another way to the same goal, points go to @vassilis.

<?php
$username = 'evercore';
$password = 'david10';
$opts = array(
    'http' => array(
        'method'  => 'GET',
        'header'  => sprintf("Authorization: Basic %s\r\nUser-Agent: Foobar!", base64_encode($username.':'.$password))
    )
);

$context = stream_context_create($opts);
libxml_set_streams_context($context);
$xml =  new SimpleXMLElement(
        "http://feeds.outgard.net/www.sportsbook.com/trends/203.xml",
        null,
        true);
var_dump($xml);