dynamic arrays and wininet in delphi?

Question

I'm using WinInet to connect and retrieve information from one of our server. I'm using the following:

indexdata: array of byte[1..5000] of byte;
infoBuffer: array [0..5000] of char;
BufferSize: DWORD;
reserved:   DWORD;
text: string;

BufferSize := Length(infoBuffer);
res := HttpQueryInfo(hHttpRequest, HTTP_QUERY_RAW_HEADERS_CRLF, @infoBuffer, BufferSize, Reserved);

Reserved := 0;
InternetReadFile(hHttpRequest, @indexdata, sizeof(indexdata), Reserved);

SetLength(text, Reserved);
CopyMemory(@text[1], @indexdata[1], Reserved);

The two array of bytes were enough up until now. Things changed. The server can return now information that can be bigger or smaller than 5000; worst yet, in InternetReadFile can return a variable size in the infoBuffer.

So i tried declaring the indexdata and infobuffer as array of byte and then using SetLength to set its length, but 2 things happened.

1) I still don't know the size of indexdata that the server will return so I cannot properly set it to, say, 100000.

2) I cannot use (as it is now) CopyMemory passing Low(indexdata) to copy indexdata to a simple string variable so I can use the data.

How do I handle this in Delphi? I can do it in C but I can't seem to be able to do it properly in Delphi.

Code is appreciated

thanks!

Answer 1

You do know that char is a Unicode character since Delphi 2009, but an ANSI character prior to Delphi 2009. In the same way, in Delphi 2009 a string is a UnicodeString, not an AnsiString.

So, when you write SetLength(text, Reserved) you do set the number of characters in text to Reserved. But the number of bytes will be 2*Reserved.

In other words, in Delphi 2009+, one char is not one byte, but two bytes.

You can get back the old behaviour by replacing all char with AnsiChar and all string with AnsiString.

Update

Since you did not post your entire code, I cannot really say what the problem is. Nevertheless, you might find it interesting to read my example usage of InternetReadFile in Delphi. See my answer to this question. It is a fully-working example of how to read a text file from the Internet using Delphi and InternetReadFile.

For your convinience, I paste my code below as well:

To read data from the Internet, use InternetReadFile function. I use the following code to read a small (one-line) text file from the Internet:

function WebGetData(const UserAgent: string; const Server: string; const Resource: string): string;
var
  hInet: HINTERNET;
  hURL: HINTERNET;
  Buffer: array[0..1023] of AnsiChar;
  i, BufferLen: cardinal;
begin
  result := '';
  hInet := InternetOpen(PChar(UserAgent), INTERNET_OPEN_TYPE_PRECONFIG, nil, nil, 0);
  try
    hURL := InternetOpenUrl(hInet, PChar('http://' + Server + Resource), nil, 0, 0, 0);
    try
      repeat
        InternetReadFile(hURL, @Buffer, SizeOf(Buffer), BufferLen);
        if BufferLen = SizeOf(Buffer) then
          result := result + AnsiString(Buffer)
        else if BufferLen > 0 then
          for i := 0 to BufferLen - 1 do
            result := result + Buffer[i];
      until BufferLen = 0;
    finally
      InternetCloseHandle(hURL);
    end;
  finally
    InternetCloseHandle(hInet);
  end;
end;

Sample usage:

WebGetData('My UserAgent', 'www.rejbrand.se', '/MyDir/update/ver.txt')

Answer 2

Instead of CopyMemory, try using the Move routine, like so:

Move(indexdata[1], text[1], reserved);

(Yes, without the @ symbols.) That should solve problem #2. As for problem #1, that's between you and the server. You need to have some way to know what the upper bound is on the size that it will return, and make your buffer at least that large. You should either be able to find this in the documentation, or call another API first that will give you the size of the incoming data.

Also put some check on there so that if it returns something larger than your buffer, it immediately raises an exception. If your buffer is overrun, you should assume that your program is under attack and react accordingly.

Answer 3

Windows functions that return a buffer of unknown size, usually return the require size of the buffer if a too small buffer is passed. This way you do not need to size in advance a buffer large enough to hold the largest data - pass a small buffer (even a zero length one), and the function will return how large the buffer should be. Then you can size your buffer and pass it again the the function to read data - check for example HttpQueryInfo() explanation in MSDN. To handle buffer of an unknown size in Delphi, you have two ways.

• Use dynamic arrays and SetLength()
• Use dynamically allocated buffers using GetMem() and FreeMem(), very much alike you would do in C.

For example:

var
  Buf: Pointer;
  BufSize: DWORD; 
  Rez: DWORD;
  ...
const
  InitialBufSize = 1024;

...
BufSize := InitialBufSize;
GetMem(Buf, BufSize);
try
  if not HttpQueryInfo(hRequest, dwInfoLevel, Buf, BufSize, lpdwIndex) then
  begin
    Rez := GetLastError;
    if Rez = ERROR_INSUFFICIENT_BUFFER then
    begin
      FreeMem(Buf, InitialBufSize);
      GetMem((Buf, BufSize);
      if not HttpQueryInfo(hRequest, dwInfoLevel, Buf, BufSize, lpdwIndex) then
        Rez := GetLastError
    end;
 ...
finally
  FreeMem(Buf, BufSize);
end;