Handling delays when retrieving files from remote server in PHP

Question

Hello,

I am working with PHP to access files and photos from remote servers. I am mostly using the file_get_contents() and copy() functions.

Sometimes accessing a small text file or photo is almost instant, but other times it seems to get "stuck" for a minute on the same exact file. And sometimes it actually causes my script to hang, and even when I stop the script Apache remains locked up for several minutes.

I'm quite willing to accept the fact that internet connections can be flaky. My concern is that I recover gracefully and that I do not crash Apache - the PHP set_time_limit() function only returns a fatal error. In addition, there is a note in the PHP manual that time spent on stream operations does not contribute to the running time of the script.

How can I recover from such connection problems and allow my script to continue? And why would this be causing Apache to hang?

Thanks, Brian

Answer 1

$options = array( 'http' => array(
      'user_agent'    => 'Firefox wannabe',
      'max_redirects' => 1,
      'timeout'       => 10,
  ) );
$context = stream_context_create( $options );
$content    = @file_get_contents( $url, false, $context );

Take a look at http://php.net/manual/en/function.stream-context-create.php and http://www.php.net/manual/en/context.http.php . The above code will set a timeout on the connection, and will allow for redirects.

This should prevent reaching the timeout.

The long delays may be caused by the network or by the remote server having a firewall denying you to grab too many files at once or by a flanky dns server or router on the path to the remote host. As a suggestion, you should cache locally the downloaded files, so on the next refresh files will be handled locally instead of the big wide net.