This is an interesting problem that I’ve not been able to solve yet.
I am writing a client that communicates across the Internet to a server. I am using the TIdTcpClient Internet Direct (Indy) component in Indy 10 using RAD Studio 2007 native personality.
To get data from the server, I issue an HTTP request using SSL over port 443 where my request details are contained in the HTTP message body. So far, so good. The code works like charm, with one exception.
There is one request that I am submitting that should produce a response of about 336 KB from the server (the HTTP response header contains Content-Length: 344795). The problem is that I am getting only 320KB back. The response, which is in XML, is clearly truncated in the middle of an XML element.
For what it’s worth, the XML is simple text. There are no special characters that can account for the truncation. My TIdTcpClient component is simply reporting that, after receiving the partial response, that the server closed the connection gracefully (which is how every response is expected to be completed, even those that are not truncated, so this is not a problem).
I can make nearly identical calls to the same server where the response is also more than a few K bytes, and all of these work just fine. One request I make returns about 850 KB, another returns about 300 KB, and so on.
In short, I encounter this problem only with the one specific request. All other requests, of which there are many, receive a complete response.
I have talked to the creator of the service, and have supplied examples of my request. He reported that the request is correct. He also told me that when he issues my same request to his server that he gets a complete response.
I’m at a loss. Either the creator of the service is mistaken, and there is actually a problem with the response on that end, or there is something peculiar about my request.
Is there a solution here that I'm missing? Note that I’ve also used a number of other read mechanisms (ReadString, ReadStrings, ReadBytes, etc) and all produce the same result, a truncation of this one specific response at the 320KB mark.
The code is probably not relevant, but I’ll include it anyway. Sorry, but I cannot include the XML request, as it includes proprietary information. (ReadTimeout is set to 20 seconds, but the request returns in about 1 second, so it's not a timeout issue.)
function TClient.GetResponse(PayloadCDS: TClientDataSet): String;
var
s: String;
begin
try
try
s := GetBody(PayloadCDS);
IdTcpClient1.Host := Host;
IdTcpClient1.Port := StrToInt(Port);
IdTcpClient1.ReadTimeout := ReadTimeout;
IdTcpClient1.Connect;
IdTcpClient1.IOHandler.LargeStream := True;
//IdTcpClient1.IOHandler.RecvBufferSize := 2000000;
IdTcpClient1.IOHandler.Write(s);
Result := IdTcpClient1.IOHandler.AllData;
except
on E: EIdConnClosedGracefully do
begin
//eat the exception
end;
on e: Exception do
begin
raise;
end;
end;
finally
if IdTcpClient1.Connected then
IdTcpClient1.Disconnect;
end;
end;