How do I zip on the fly and stream to Response.Output in real time?

Question

I am trying to use the following code: I get a corrupted zip file. Why? The file names seem OK. Perhaps they are not relative names, and that's the problem?

      private void trySharpZipLib(ArrayList filesToInclude)
    {
        // Response header
        Response.Clear();
        Response.ClearHeaders();
        Response.Cache.SetCacheability(HttpCacheability.NoCache);
        Response.StatusCode = 200; // http://community.icsharpcode.net/forums/p/6946/20138.aspx
        long zipSize = calculateZipSize(filesToInclude);
        string contentValue = 
            string.Format("attachment; filename=moshe.zip;"
                          ); // + " size={0}", zipSize);
        Response.ContentType = "application/octet-stream"; //"application/zip"; 
        Response.AddHeader("Content-Disposition", contentValue);
        Response.Flush();

        using (ZipOutputStream zipOutputStream = new ZipOutputStream(Response.OutputStream) ) 
        {
            zipOutputStream.SetLevel(0);

            foreach (string f in filesToInclude)
            {
                string filename = Path.Combine(Server.MapPath("."), f);
                using (FileStream fs = File.OpenRead(filename))
                {
                    ZipEntry entry =
                        new ZipEntry(ZipEntry.CleanName(filename))
                            {
                                DateTime = File.GetCreationTime(filename),
                                CompressionMethod = CompressionMethod.Stored,
                                Size = fs.Length
                            };
                    zipOutputStream.PutNextEntry(entry);

                    byte[] buffer = new byte[fs.Length];
                    // write to zipoutStream via buffer. 
                    // The zipoutStream is directly connected to Response.Output (in the constructor)
                    ICSharpCode.SharpZipLib.Core.StreamUtils.Copy(fs, zipOutputStream, buffer); 
                    Response.Flush(); // for immediate response to user
                } // .. using file stream
            }// .. each file
        }
        Response.Flush();
        Response.End();
    }

Answer 1

Try adding the following header.

Response.AddHeader("Content-Length", zipSize);

I know that was causing me issues before.

Edit:

These other 2 may help as well:

Response.AddHeader("Content-Description", "File Transfer");
Response.AddHeader("Content-Transfer-Encoding", "binary");

Answer 2

Have you tried flushing the ZipOutputStream before flushing the response? Can you save the zip on the client and test it in a zip utility?

Answer 3

Not quite sure how to do this in ASP.NET (haven't tried it before), but in general if the HTTP client supports HTTP v1.1 (as indicated by the version of its request), the server can send a 'Transfer-Encoding' response header that specifies 'chunked', and then send the response data using multiple data blocks as they become available. This allows for real-time streaming of data where you don't know the final data size ahead of time (and thus cannot set the 'Content-Length' response header). Have a look at RFC 2616 Section 3.6 for more details.

Answer 4

Boy, that's a lot of code! Your job would be simpler using DotNetZip. Assuming a HTTP 1.1 client, this works:

Response.Clear();
Response.BufferOutput = false;
string archiveName= String.Format("archive-{0}.zip", DateTime.Now.ToString("yyyy-MMM-dd-HHmmss"));
Response.ContentType = "application/zip";
Response.AddHeader("content-disposition", "filename=" + archiveName);  
using (ZipFile zip = new ZipFile())
{
    // filesToInclude is a IEnumerable<String> (String[] or List<String> etc)
    zip.AddFiles(filesToInclude, "files");
    zip.Save(Response.OutputStream);
}
// Response.End();
HttpContext.Current.ApplicationInstance.CompleteRequest();

If you want to password-encrypt the zip, then before the AddFiles(), insert these lines:

    zip.Password = tbPassword.Text; // optional
    zip.Encryption = EncryptionAlgorithm.WinZipAes256; // optional

If you want a self-extracting archive, then replace zip.Save() with zip.SaveSelfExtractor().

---

Addendum; some people have commented to me that DotNetZip is "no good" because it creates the entire ZIP in memory before streaming it out. This isn't the case. When you call AddFiles, the library creates a list of entries - objects that represent the state of the things to be zipped up. There is no compression or encryption done until the call to Save. If you specify a stream to the Save() call, then all the compressed bytes get streamed directly to the client.

In the SharpZipLib model, it's possible to create an entry, then stream it out, then create another entry, and stream it out, and so on. With DotNetZip your app creates the complete list of entries first, and then streams them all out. Neither approach is necessarily "faster" than the other, though for long lists of files, say 30,000, the time-to-first-byte will be faster with SharpZipLib. On the other hand I would not recommend dynamically creating zip files with 30,000 entries.

Some people have the case where they have zip content that is "mostly the same" for all users, but there's a couple of files that are different for each one. DotNetZip is good at this, too. You can read in a zip archive from a filesystem file, update a few entries (add a few, remove a few, etc), then save to Response.OutputStream. In this case DotNetZip does not re-compress or re-encrypt any of the entries that you haven't changed. Much faster.

Of course DotNetZip is for any .NET app, not only ASP.NET. So you can save to any stream. Ok, that's enough of that. If you want more info, check out the site or post on the dotnetzip forums.

Answer 5

For those who would miss the ZipOutputStream of SharpZipLib, here is a simple code that makes it possible to use DotNetZip the "regular .NET stream way".

However, take note that it is inefficient compared to a real on-the-fly streaming solution like SharpZipLib does, as it uses an internal MemoryStream before actually calling the DotNetZip.Save() function. But unfortunately, SharpZibLib does not sports EAS Encryption yet (and certainly never). Let's hope that Cheeso will add this functionality in dotNetZip any time soon ? ;-)

/// <summary>
/// DotNetZip does not support streaming out-of-the-box up to version v1.8.
/// This wrapper class helps doing so but unfortunately it has to use
/// a temporary memory buffer internally which is quite inefficient
/// (for instance, compared with the ZipOutputStream of SharpZibLib which has other drawbacks besides).
/// </summary>
public class DotNetZipOutputStream : Stream
{
    public ZipFile ZipFile { get; private set; }

    private MemoryStream memStream = new MemoryStream();
    private String nextEntry = null;
    private Stream outputStream = null;
    private bool closed = false;

    public DotNetZipOutputStream(Stream baseOutputStream)
    {
        ZipFile = new ZipFile();
        outputStream = baseOutputStream;
    }

    public void PutNextEntry(String fileName)
    {
        memStream = new MemoryStream();
        nextEntry = fileName;
    }

    public override bool CanRead { get { return false; } }
    public override bool CanSeek { get { return false; } }
    public override bool CanWrite { get { return true; } }
    public override long Length { get { return memStream.Length; } }
    public override long Position
    {
        get { return memStream.Position; }
        set { memStream.Position = value; }
    }

    public override void Close()
    {
        if (closed) return;

        memStream.Position = 0;
        ZipFile.AddEntry(nextEntry, Path.GetDirectoryName(nextEntry), memStream);
        ZipFile.Save(outputStream);
        memStream.Close();
        closed = true;
    }

    public override void Flush()
    {
        memStream.Flush();
    }

    public override int Read(byte[] buffer, int offset, int count)
    {
        throw new NotSupportedException("Read");
    }

    public override long Seek(long offset, SeekOrigin origin)
    {
        throw new NotSupportedException("Seek");
    }

    public override void SetLength(long value)
    {
        memStream.SetLength(value);
    }

    public override void Write(byte[] buffer, int offset, int count)
    {
        memStream.Write(buffer, offset, count);
    }
}