tags:

views:

222

answers:

5

I've been trying to connect to my own site using both CURL and the PHP file_get_contents() function to get source of my webpage without success. I am running the PHP script on the same server that I am trying to get the HTML source from. CURL doesn't return any errors, not even when using curl_error(), and the PHP file_get_contents() function returns the following:

Warning: file_get_contents([sitename]) [function.file-get-contents]: failed to open stream: Connection refused in [file path] on line 19.

I've no idea why this is. Why would a server actively refuse this connection? How can I stop it?

Thanks

EDIT:

For reference here is my (cURL) code:

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://www.mydomain.co.uk');
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, '');
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2');
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Host: www.mydomain.co.uk')); 

$rawHTML = curl_exec($ch);
$err = curl_error($ch);
curl_close($ch);

print $err;
print 'HTML: ' . $rawHTML;
+3  A: 

Have a look at your firewall settings, they may be a little too strict. What happens if you log in and

telnet localhost 80

or the equivalent for your os of choice? And try the same not with localhost but the full ip of your server. Only if it succeeds, you have a curl/php problem.

edit: ok, so connection to localhost works, using file_get_contents("http://localhost/");.

This means that you can access you site through localhost, but you need to override the Host: field sent with the request. This is not exactly normal usage of cURL, but you may try:

curl_setopt(CURLOPT_HTTPHEADER,array('Host: yourdomain.com'));

while requesting URL http://127.0.0.1/. I wonder if this will be understood by curl but you can give it a shot.

edit^2: If this does not work to trick cURL, just open your own socket connection and make your own request:

$ip = '127.0.0.1';
$fp = fsockopen($ip, 80, $errno, $errstr, 5);
$result = '';
if (!$fp) {
    echo "$errstr ($errno)<br />\n";
} else {
    $out = "GET / HTTP/1.1\r\n";
    $out .= "Host: www.exampl.com\r\n";
    $out .= "Connection: Close\r\n\r\n";
    fwrite($fp, $out);
    while (!feof($fp)) {
        $result .= fgets($fp, 128);
    }
    fclose($fp);
}

(this is an adaption from a php.net example)

mvds
+1 - Normally `localhost` would work, but often using the actual server name or public IP would not. Many firewalls/routers will not allow requests that essentially "loop back" through the router.
Eric Petroelje
I'm afraid I don't have telnet access to the web server. Is there anyway I can emulate/test this using PHP commands?
CJD
err.. that's basically what you are doing ;-) Try to connect to localhost (or `127.0.0.1`, or `0.0.0.0`, all the same) and see what happens. If you're in a shared hosting environment you'll get some error response, but at least you know you have connectivity. So for example `file_get_contents("http://127.0.0.1/");`
mvds
Ok, got you. Following your suggestion, I get a blank page. No error messages, no output, nothing (error reporting is on).
CJD
so you get no connection error this time? could be your webserver will do that when given no hostname; what happens if you access it from home, using it's ip address? i.e. point your browser at `http://1.2.3.4/`
mvds
Sorry, scratch that. I do get a response (I wasn't printing the output, doh!), however it's not promising, all it says is "This domain name has been reserved for future use". I suspect this is the result of me using shared hosting.
CJD
yes exactly, but it tells us that you *can* connect to `localhost`. So now the goal is to connect to `localhost` and have curl or php send the right (somewhat fake) `Host:` header field. One way would be to use a `/etc/hosts` file, but I presume chances are very slim you could do that (and besides it may be a little risky).
mvds
Thanks for answering. Yes, unfortunatly I don't have access to the /etc/hosts file on the web server. I think I might just have to accept that it's not going to be possible to do this.
CJD
see my updated answer. If this doesn't work there's even more tricks; don't give up, you will have learned a lot by the time you get it to work.
mvds
Still no luck, I've put the cURL code I'm using in my original question for reference. All I'm getting is a blank page.
CJD
just added a socket example; you can skip prefab stuff and just connect yourself. Watch out with the precise form of the request, since it's easy to break!
mvds
No luck I'm afraid, still a blank output. Running out of ideas yet? ;)
CJD
you did `echo $result;`? no error whatsoever? `error_reporting(~0)` at the top of your file?
mvds
Ah I'm having one of those days. Printing the result does the trick! Thanks so much for your help!
CJD
great to see you solved the mystery ;-)
mvds
Are there any content encoding issues I should be aware about when using sockets? I'm getting exactly the same output as I would by visiting the page in the browser but json_decode() isn't working. Weirdly, it's returning the same value that I'm passing to it.
CJD
no issues I think, you get the raw bytes from the socket, but that *does* include the HTTP response header. Did you cut that off? try `list($headers,$body)=explode("\r\n\r\n",$result,2);`
mvds
Yea I've cut the headers off. Hmm I'll do some more investigating tomorrow. Thanks again for all your help.
CJD
watch out for chunked encoding btw, that'll give some noise in you content if not handled properly (some short hex numbers, can't miss)
mvds
Managed to fix it. For some reason a bunch of special chars were in the string, this code in the comments of the PHP json_decode page did the trick: preg_replace('/[\x00-\x1F\x80-\xFF]/', '', $json);
CJD
well done. But just stripping characters from your input is hardly a solution; you should look into the encoding issue, using `iconv` or `mb_convert_encoding` to test and/or convert encodings (and then maybe open another question ;-)
mvds
A: 

I have had this problem before when using a shared server with a hosting company, as many features are turned off. Are you on a shared server/do you get your hosting for free?

Check if cURL is enabled on your server too in your PHP config.

<?php

// Show all information
phpinfo();

?>

If the data you are trying to access is on the same server, you could use fopen:

$myFile = "testFile.txt";
$fh = fopen($myFile, 'r');
$theData = fread($fh, 5);
fclose($fh);
echo $theData;

http://php.net/manual/en/function.fopen.php

Joel Kennedy
CURL is enabled as it works with other sites. And indeed, I'm using shared hosting which I guess is why it's not working.
CJD
Yes I seem to remember having the exact same problem on my shared host. I don't know if you have already considered it, but you could use fopen instead? I have edited my post above.
Joel Kennedy
Afraid not, as I'm after the output, not the content.
CJD
Hmm, the only other method I can see is using an include if it is a PHP file, and then call the functions directly. I really do think you need cURL to be able to do what you are saying. Perhaps requesting it to be turned on by your hosting provider?
Joel Kennedy
Hmm, I might be able to try including it. It would be a bit of a bodge though. Thanks for your help anyway.
CJD
No problem, sorry I couldn't help further.
Joel Kennedy
A: 

Make sure that you have allow_url_fopen turned on from your php.ini.

Sarfraz
A: 

Possibly a stupid question, but you don't have an .htaccess password set do you? If your browser's storing it you might forget, but the PHP server doesn't have it but will still need it to access the site. Almost certainly not, but maybe worth suggesting.

Richeh
Thanks for the suggestion, but alas no, I've got no password protection on the root URL.
CJD
+1  A: 

Most possibly because your site is pointed to a public ip, which in turn is mapped to an internal IP like mvds pointed out.

www.domain.com = 234.234.234.234

server ip : 10.0.0.1

firewall maps 234.234.234.234 -> 10.0.0.1 from outside networks, but not from internal traffic.

Hence, you need to connect to your machine with the local IP, or localhost (127.0.0.1) but still maintaining the host-header (www.domain.com).

Your options are:

  1. Make your provider setup correct routing for their external IPs in their firewall. It's doable but often missed since it's rarely needed. One argument for this to work is because you cannot access other sites that resides on the same network segment as you today.
  2. Tell your provider to add www.domain.com -> 127.0.0.1 in the hosts-file on your server
  3. Use your own socket-code to "fake" the host-header while still connecting to localhost. There are plenty of example classes for this in PHP and mvds gave you an example already.
  4. Find another way to fetch the information. It's on the same server, isn't it? Fetching it by http seems redundant...
jishi
Thanks for the detailed explanation and advice.
CJD
ad 4) a really ugly (yet effective) hack is to set `$_GET[...]=...` to whatever parameters you need and then `include` a file.
mvds