Uploading files in WCF services approaches

Question

I have WCF service that I can upload files.

XElement upload = service.UploadFiles(id, File.ReadAllBytes(filePath));

This works, but I am just wondering if it is the best way to upload each of them one by one when you have many (about a thousand) small files (15~20K). How can we compare "Chunky" versus "Chatty" approaches?

1) Is it better to send each of them, or zip them all in one file to save the latency time for acknowledgment of each of them.

2) Having said that, I need to increase the maximum size of array or other settings as I want to zip them all together. Is this recommended? Cause I read that this acknowledgment for each of them could be more expensive comparing to the sending one big file and one acknowledgment at the end?

thanks,

Answer 1

As you seem to know, you'll generally want your services to be "chunky" as opposed to "chatty". The answer in your specific scenario will depend on:

• How large the total of all the files is.
• If and how you use the acknowledgments you're currently getting. If you need to update a UI a each file is uploaded you'll find it difficult to keep level of detail if you send them all at once.
• What type of reliability/retryability you need. Right now, if your service or connection dies you can simply resend the files that didn't get sent. If you batch all the files together you'll have to start all over again, which may or may not be a big issue.

If you can, you might want to look at the support for sending Streams with WCF:

• http://msdn.microsoft.com/en-us/library/ms731913.aspx
• http://www.haveyougotwoods.com/archive/2008/04/14/wcf-message-streaming.aspx

By sending a Stream you get the size benefits of zipping up the entire batch of files, while still retaining the ability to know your progress within the upload.

I'm not sure of any specific issues regarding increasing the maximum message size, beyond the impact it has on memory usage. Lastly, I would say that you should be zipping up these files regardless of if you send them all together.