curl halts script execution

Question

my script uses curl to upload images to smugsmug site via smugsmug api. i loop through a folder and upload every image in there. but after 3-4 uploads, curl_exec would fail, stopped everything and prevent other images from uploading.

$upload_array = array(
    "method" => "smugmug.images.upload",
    "SessionID" => $session_id,
    "AlbumID" => $alb_id,
    "FileName" => zerofill($n, 3) . ".jpg",
    "Data" => base64_encode($data),
    "ByteCount" => strlen($data),
    "MD5Sum" => $data_md5);
$ch = curl_init();
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0); 
curl_setopt($ch, CURLOPT_POSTFIELDS, $upload_array);
curl_setopt(
    $ch, CURLOPT_URL, 
    "https://upload.smugmug.com/services/api/rest/1.2.2/");
$upload_result = curl_exec($ch); //fails here
curl_close($ch);

updated: so i added logging into my script. when it does fail, the logging stops after fwrite($fh, "begin curl\n");

fwrite($fh, "begin curl\n");
$upload_result = curl_exec($ch);
fwrite($fh, "curl executed\n");
fwrite($fh, "curl info: ".print_r(curl_getinfo($ch,true))."\n");
fwrite($fh, "xml dump: $upload_result \n");
fwrite($fh, "curl error: ".curl_error($ch)."\n");

i also

curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 60*60);

Answer 1

Not sure what the issue is... What is in the response when it fails? What do the system and apache logs say?

Now if i were you i wouldnt use curl_init() and curl_close() in the loop. instead i would init before the loop, and close after the loop - then within the loop itsef i would use curl_set_opt to set the url and differing parameters and just call curl_exec(). It may even be its a matter of all these handles exceeding some kind of system limit or something. If you need/want to use multiple connections you could use curl_multi or write some management functions/class to manage multiple handles.

Answer 2

We may need more info before we can help, but it sounds like it could be a timeout issue.

Turn on error reporting or check your error logs to see if anything is being raised.

Try setting a long cURL timeout with CURLOPT_TIMEOUT

Also check that your script timeout is sufficient or increase with set_time_limit()

Answer 3

You could try whether outputting curl_error() directly works:

$upload_result = curl_exec($ch); //fails here

$error = curl_error($ch();    
if ($error) echo "CURL Error: $error";

curl_close($ch);

If that doesn't help, check your phpinfo(); to see whether error reporting is turned off globally (look for the display_errors setting).

Answer 4

CURL includes the 'multi' (for multiple-resources) options for when one is dealing with multiple high-latency requests (such as uploading images).

See: http://www.php.net/manual/en/function.curl-multi-exec.php and the entire library of 'multi' functions described here: http://php.net/manual/en/book.curl.php

For a complete example of the multiple resource section of the library, see: http://www.developertutorials.com/blog/php/parallel-web-scraping-in-php-curl-multi-functions-375/

Answer 5

1- Force Curl to tell you a bit more about what it does

curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_HEADER, true);

2- If you are not in safe mode, make sure PHP displays errors by putting this at the beginning of your script:

<?php
ini_set('display_errors', '1');
error_reporting(E_ALL);

3- You can also try to run your script in CLI mode.

4- Finally, still if you are not in safe mode, you can try to directly run the curl binary using exec().

<?php

$curl_str = "curl -k -o /my/path/curl_output.log -d 'var1=".$value1."&var2=".$value2."& etc...' https://upload.smugmug.com/services/api/rest/1.2.2/";
$r = exec($curl_str);