Curl RETURNTRANSFER data ?timing? problem and multiple identical POSTs

Question

Hello,

I am running into a sporadic problem with curl's RETURNTRANSFER option. I am not sure if I have overlooked something in my code that is causing it, or if it is just not well documented and I am unaware of the RETURNTRANSFER workings.

I am using curl (via php) to send xml data via POST to an external listener. If successful, the listener responds and returns the same xml, with one or two fields populated that I sent through empty. eg. <Status /> becomes <Status>Accepted</Status>. A rough size of the full xml POST - [upload_content_length] => 1414. A unique ID (generated on my side) is sent, if the listener receives a duplicate ID, it declines the xml data and responds with a plain text error message.

This works 99.9% of the time.

The curl code is :

$header[] = "Content-Type: text/xml";
$header[] = "Content-length: ".strlen($post_string);
$x = curl_init("https://xyz.com");
curl_setopt($x, CURLOPT_HEADER, 0);
curl_setopt($x, CURLOPT_HTTPHEADER, $header);
curl_setopt($x, CURLOPT_POST, 1);
curl_setopt($x, CURLOPT_POSTFIELDS,$xmldata);
curl_setopt($x, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($x, CURLOPT_REFERER, "https://mypage.com");
curl_setopt($x, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($x, CURLOPT_PORT, 443);
curl_setopt($x, CURLOPT_SSL_VERIFYPEER, 0); //to allow self-signed SSL cert
curl_setopt($x, CURLOPT_SSL_VERIFYHOST, 0); //to allow self-signed SSL cert
curl_setopt($x, CURLOPT_TIMEOUT, 30);
curl_setopt($x, CURLOPT_FORBID_REUSE, TRUE);
curl_setopt($x, CURLOPT_FRESH_CONNECT, TRUE);
curl_setopt($x, CURLOPT_BUFFERSIZE, 64000);
curl_setopt($x, CURLOPT_VERBOSE, 1);
$listenerresponse = curl_exec($x);
$info = curl_getinfo($x);
curl_close($x); 

Recently I have been picking up errors where the listener errors on duplicate IDs.

My logs show the timeline as this ::

1. curl POST of unique id "12345678"

2. listener responds "Duplicate ID" at 2010-10-13 17:27:59

3. listener responds "Accepted" 2010-10-13 17:28:06

Initially I though a previous ID had been cached somehow and was being sent erroneously, but this is not the case. The listener receives the correct ID. The listener logs show it the other way around, with a slightly differing chronology:

1. Listener responds "Accepted" at 2010-10-13 17:27:48

2. Listener responds "Duplicate ID" at 2010-10-13 17:27:58

From my understanding, I would expect the curl thread to wait for the response, and not send another request. Am I right in presuming that the problem is network related - possible packets dropping/something similar? Any suggestions on how to prevent this happening would be most appreciated. Ideally I would like curl to send and wait for the full response before sending off another request... Is there an option to specify this? Please help :-)

Thank you for your attention.

S.

Answer 1

By default CURL will block (e.g. wait) for a request to complete before returning from the curl_exec()ca. . You don't appear to be using multi-curl (which allows for multiple parallel requests), so there shouldn't be any way for a packet from request A to get mixed in with packets from request B,C,... Request A will complete and return before request B is fired off.

Do you have control over the listener script? You might want to include a token you generate with each request, on top of that "unique" ID parameter, and have the listener spit it out in any error messages, so you can track down exactly which request caused the duplicate ID errors.