How to retrieve a file from Internet via HTTP?

Question

I want to download a file from Internet and InternetReadFile seem a good and easy solution at the first glance. Actually, too good to be true. Indeed, digging a bit I have started to see that actually there are a lot of issues with it. People are complaining about all kinds of problems when using this code.

Problems could appear because:

• the application freezes temporarily until the HTTP server responds
• the application freezes temporarily because the Internet connections breaks
• the application locks up because the HTTP server never responds
• the InternetOpen (I just discovered this recently) MUST be called only once during application life time

I could not find a complete example about how to use it properly and robustly. Does anybody have an idea about how to implement it in a separate thread and with a time out? There is another SIMPLE way to robustly download a file from Internet. Though I don't want to complicate my life with very large libraries like Jedi or even Indy.

function GetFileHTTP (const fileURL, FileName: String): boolean;
CONST
  BufferSize = 1024;
VAR
  hSession, hURL: HInternet;
  Buffer: array[1..BufferSize] of Byte;
  BufferLen: DWORD;
  f: File;
  sAppName: string;
begin
//  result := false;
 sAppName := ExtractFileName(Application.ExeName) ;
 hSession := InternetOpen(PChar(sAppName), INTERNET_OPEN_TYPE_PRECONFIG, nil, nil, 0) ;  { be aware that InternetOpen  need only be called once in your application!!!!!!!!!!!!!! }
 TRY
  hURL := InternetOpenURL(hSession, PChar(fileURL), nil, 0, 0, 0) ;
  TRY
   AssignFile(f, FileName) ;
   Rewrite(f, 1) ;
   REPEAT
    InternetReadFile(hURL, @Buffer, SizeOf(Buffer), BufferLen);
    BlockWrite(f, Buffer, BufferLen)
   UNTIL BufferLen = 0;
   CloseFile(f) ;
   Result:= True;
  FINALLY
   InternetCloseHandle(hURL)
  end
 FINALLY
  InternetCloseHandle(hSession)
 END;
END;

Edit: This functions checks if Internet connection is available. It seems to work on Win98 also.

{  Are we connected to the Internet? }
function IsConnectedToInternet: Boolean;                                        { Call SHELL32.DLL for Win < Win98 otherwise call URL.dll }
var InetIsOffline: function(dwFlags: DWORD): BOOL; stdcall;
begin
 Result:= FALSE;
 if IsApiFunctionAvailable('URL.DLL', 'InetIsOffline', @InetIsOffline)
 then Result:= NOT InetIsOffLine(0)
 else
   if IsApiFunctionAvailable('SHELL32.DLL', 'InetIsOffline', @InetIsOffline)
   then Result:= NOT InetIsOffLine(0)
end;

I am using Delphi 7. Many thanks.

---

Edit:

Losing customers because the application hangs at the first start up is the perfect recipe for losing money.

Writing your code to be Microsoft platform dependent is bad. You never know if the customer has the IE version x.x installed.

Installing stuff into a user's computer is like playing with guns. It will backfire.

(see more about this here: http://thesunstroke.blogspot.com/2010/06/programmig-like-there-is-no-ms-windows.html)

Answer 1

I basically do the same as you do. For me it works fairly flawlessly.

The only differences between my code and your code is I have an INTERNET_FLAG_RELOAD parameter to force a download from the file and not the cache. You can try that and see if it works better:

  hURL := InternetOpenURL(hSession, PChar(fileURL), nil, 0, INTERNET_FLAG_RELOAD, 0) ; 

Also check for an internet connection before downloading. Do this:

  dwConnectionTypes := INTERNET_CONNECTION_MODEM
                 + INTERNET_CONNECTION_LAN
                 + INTERNET_CONNECTION_PROXY;
  InternetConnected := InternetGetConnectedState(@dwConnectionTypes, 0);
  if InternetConnected then ...

Answer 2

Here's some sample code that uses Indy. This code is for Delphi 2010 (with Indy 10?), but the code for Delphi 7 would be similar. I've used Indy for years with D7 and have been very happy with it. I think in D7 we use Indy 9. Check if you need to download a new version...

You can use OnWork and OnWorkBegin to add a progress meter if you need to.

This code I excerpted from a bigger piece, editing it a bit. I did not try compiling it, but it will give you a good starting place.

function Download( const aSourceURL: String;
                   const aDestFileName: String;
                   out   aDownloadResult: TDownloadResult;
                   out   aErrm: String): boolean;
var
  Stream: TMemoryStream;
  IDAntiFreeze: TIDAntiFreeze;
begin
  aDownloadResult := DROther;
  Result := FALSE;
  fIDHTTP := TIDHTTP.Create;
  fIDHTTP.HandleRedirects := TRUE;
  fIDHTTP.AllowCookies := FALSE;
  fIDHTTP.Request.UserAgent := 'Mozilla/4.0';
  fIDHTTP.Request.Connection := 'Keep-Alive';
  fIDHTTP.Request.ProxyConnection := 'Keep-Alive';
  fIDHTTP.Request.CacheControl := 'no-cache';
  IDAntiFreeze := TIDAntiFreeze.Create;

  Stream := TMemoryStream.Create;
  try
    try
      fIDHTTP.Get(aSourceURL, Stream);
      if FileExists(aDestFileName) then
        DeleteFile(PWideChar(aDestFileName));
      Stream.SaveToFile(aDestFileName);
      Result := TRUE;
      aDownloadResult :=drSuccess;
    except
      On E: Exception do
        begin
          Result := FALSE;
          aErrm := E.Message + ' (' + IntToStr(fIDHTTP.ResponseCode) + ')';
        end;
    end;
  finally
    Stream.Free;
    IDAntiFreeze.Free;
    fIDHTTP.Free;
  end;
end;  { Download }

Answer 3

Instead of fiddling with the WinAPI, the ExtActns unit provides just what you need for downloading to a file.

procedure TMainForm.DownloadFile(URL: string; Dest: string); 
var 
  dl: TDownloadURL; 
begin 
  dl := TDownloadURL.Create(self); 
  try 
    dl.URL := URL; 
    dl.FileName := Dest; 
    dl.ExecuteTarget(nil); //this downloads the file 
    dl.Free; 
  except 
    dl.Free; 
  end; 
end; 

Under the hood, it uses URLDownloadToFile from the URLMon library - which is part of IE, and therefore part of Windows.

TDownloadURL doesn't handle any timeout for you - it doesn't look like such a thing is supported in URLMon at all, although there could be some default timeout that causes the call to fail - but you could use the OnProgress event on TDownloadURL to get notified when something happens, and then do something in another thread if it's been too long since the last callback.

Answer 4

My personal favorite is using the WebHttpRequest component from importing the "Microsoft WinHTTP Services" type library: http://yoy.be/item.asp?i142

var
  w:IWebHttpRequest;
  f:TFileStream;  
  os:TOleStream;
begin 
  w:=CoWebHttpRequest.Create;
  w.Open('GET',SourceURL,false);
  w.Send(EmptyParam);
  os:=TOleStream.Create(IUnknown(w.ResponseStream) as IStream);
  f:=TFileStream.Create(DestinationFilePath,fmCreate);
  os.Position:=0;
  f.CopyFrom(os,os.Size);
  f.Free;
  os.Free;
  w:=nil;
end;

Answer 5

I recommend Synapse. It's small, stable and easy-to-use (no need of any external libraries).

Example from httpsend.pas

function HttpGetText(const URL: string; const Response: TStrings): Boolean;
var
  HTTP: THTTPSend;
begin
  HTTP := THTTPSend.Create;
  try
    Result := HTTP.HTTPMethod('GET', URL);
    if Result then
      Response.LoadFromStream(HTTP.Document);
  finally
    HTTP.Free;
  end;
end;

Answer 6

Solved using improved version of the above code. (it still does not solve all issues - MS does not actually implemented full support for server time out)

http://stackoverflow.com/questions/3135003/the-connection-does-not-timeout-while-downloading-file-from-internet