Upload Multiple FTP Files

Question

Requirement, upload 1500 jpg images every night, the below code opens and closes a connection many times, I'm wondering if there is a better way.

...this is a code snippet, so there are variables here that are defined elsewhere

Dim picClsRequest = DirectCast(System.Net.WebRequest.Create(ftpImagePath), System.Net.FtpWebRequest)
Dim picClsStream As System.IO.Stream

Dim picCount As Integer = 0
For i = 1 To picPath.Count - 1
    picCount = picCount + 1
    log("Sending picture (" & picCount & " of " & picPath.Count & "):" & picDir & "/" & picPath(i))
    picClsRequest = DirectCast(System.Net.WebRequest.Create(ftpImagePath & "/" & picPath(i)), System.Net.FtpWebRequest)
    picClsRequest.Credentials = New System.Net.NetworkCredential(ftpUsername, ftpPassword)
    picClsRequest.Method = System.Net.WebRequestMethods.Ftp.UploadFile
    picClsRequest.UseBinary = True
    picClsStream = picClsRequest.GetRequestStream()

    bFile = System.IO.File.ReadAllBytes(picDir & "/" & picPath(i))
    picClsStream.Write(bFile, 0, bFile.Length)
    picClsStream.Close()

Next

Some comments:

Yes, I know picCount is redundant...It was late at night.

ftpImagePath, picDir, ftpUsername, ftpPassword are all variables

Yes, this is unencrypted

This code works fine, I'm looking to optimize

Related Question: http://stackoverflow.com/questions/1772654/ftp-upload-multiple-files-without-disconnect-using-net

Answer 1

You could take a look at sending the files asynchronously if you want to send more than one file at a time (if the order isn't important). Have a look at the various Begin* and End* methods such as FtpWebRequest.BeginGetResponse and FtpWebRequest.EndGetResponse etc.

Additionally you could look into the FtpWebRequest.KeepAlive property, which is used to keep a connection open/cached for reuse.

Hmmm, you could also try creating one giant tar file and send the single file in one stream with one connection ;)

Answer 2

Is it possible to zip the files in different bunches, for example create 15 zip files, each having 100 images and then upload the zip, that way it will be more faster and more efficient.

There are free libraries to create zip dynamically (for example sharpZipLib)

Answer 3

Use Multithreading - Open 3-4 FTP connections at a time and upload files in parallel.

Answer 4

What about using some one of third party FTP client libraries? Most of them does not try to hide that FTP is not a stateless protocol (unlike FtpWebRequest).

Following code uses our Rebex FTP/SSL but theare are also many others.

// create client, connect and log in 
Ftp client = new Ftp();
client.Connect("ftp.example.org");
client.Login("username", "password");
client.ChangeDirectory("/ftp/target/fir");

foreach (string localPath in picPath)
{
   client.PutFile(localPath, Path.GetFileName(localPath));
}

client.Disconnect();

Or (if all your source files are in the same folder):

// create client, connect and log in 
Ftp client = new Ftp();
client.Connect("ftp.example.org");
client.Login("username", "password");

client.PutFiles(
  @"c:\localPath\*", 
  "/remote/path",
  FtpBatchTransferOptions.Recursive,
  FtpActionOnExistingFiles.OverwriteAll);

client.Disconnect();