How to transfer a large zip file(50MB) using a WCF Service thorugh SOAP to aby client?

Question

I have a WCF Service that returns a byte array with a ZIP file (50MB) to any client that requests it. If the zip is very small (say 1MB), the soap response is coming from WCF with the byte array embedded in it.But the response size is very huge even for a 1MB file. If i try the 50MB file to get transferred the Service gets hanged and throws out of memory exception, because the SOAP response becomes huge in size.

1. What is the best option available with WCF/Web Service to trasnfer large files (mainly ZIP format) as iam sending back byte array. Instead of that any good approach for sending back the file?

2. Whether WCF/Web Service is best way to transfer large files to any client or any other better option/technology avlbl so that interoperability and scalability for 10000 users can be achieved?

My Code is below

        String pathfordownload = @"D:\New Folder.zip";
        FileStream F2D = new FileStream(pathfordownload, FileMode.Open,FileAccess.Read);
        BinaryReader binReader = new BinaryReader(F2D);
        binReader.BaseStream.Position = 0;
        byte[] binFile = binReader.ReadBytes(Convert.ToInt32 (binReader.BaseStream.Length));
        binReader.Close();
        return binFile;

A working piece/real piece of information will be really helpful as iam struggling with all the data available in google and no good results for last 1 week. Thanks in advance

Answer 1

You can transfer a Stream through WCF and then you can send (almost) limitless length files.

Answer 2

Hello Padman,

I've faced the exact same problem. The out of memory is inevitable because you are using Byte arrays.

What we did is to flush the data on the hard drive, so in stead of being limited by your virtual memory your capacity for concurrent transactions is the HD space.

Then for transfer, we jut placed the file on the other computer. Of course in our case it was a server to server file transfer. If you want to de decoupled form the peer, you can use a file download in http.

So instead than responding with a file, your service could respond with a http url to the file location. Then when the client has successfully downloaded form the server with a standard HttpRequest or WebClient it calls a method to delete the file. In SOAP that could be Delete(string url), in REST that would be delete method on the resource.

I hope this makes sense to you. The most importnat part of this is to understand that in a scalable software especially if you are looking at 10000 clients (concurrent?) is that you may not use resources that are limited, like memory streams or byte arrays. But rather rely on large and easily expandable resources like a hard drive partition that coule eventually be on a SAN and IT could grow the partition as needed.