Puzzling .Net C# DeflateStream problem

Question

Hi there,

I wonder if anyone can shed some light on an issue that is driving me nuts:

I am writing a compression decompression test class. To test it, I am serializing a dataset to a memory stream, compressing it, and uncompressing it and comparing the results.

Compression is fine, but uncompression is where it hits the dirt. This is the decompress function:

    public static Stream GetUncompressedStreamCopy(Stream inStream)
    {
      Stream outStream = new MemoryStream();

      inStream.Position = 0;

      DeflateStream uncompressStream = new DeflateStream(inStream, 
        CompressionMode.Decompress, true);

      byte[] buffer = new byte[65536];

      int totalread = 0;
      int bytesread = 0;


      do {
        bytesread = uncompressStream.Read(buffer, 0, buffer.Length);
        totalread += bytesread;
        outStream.Write(buffer, 0, bytesread);
        Console.WriteLine("bytesRead: [{0}]\t outStream.Length [{1}]",
        bytesread, outStream.Length);
      } while (bytesread > 0);


      Console.WriteLine("total bytes read [{0}]", totalread);
      outStream.Flush();
      return outStream;
}

With a buffer of size 65536 the decompressed stream always returns one byte less than it was uncompressed.

Now this brings me to the second issue which I am battling with. With some buffer sizes, uncompressStream.Read returns 0 even though there is still compressed data left to extract.

For these cases, deflateStream.Read(s) only once in the do{} loop and then returns an uncompressed stream equal to buffersize, if you increase the buffersize by a single byte all is well (except for the missing byte).

Output for buffersize of 65536: (Original uncompressed data is 207833)

bytesRead: [65536]       outStream.Length [65536]
bytesRead: [65536]       outStream.Length [131072]
bytesRead: [58472]       outStream.Length [189544]
bytesRead: [18288]       outStream.Length [207832]
bytesRead: [0]           outStream.Length [207832]
total bytes read [207832]

buffersize of 189544 (Some magic number where the code tanks)

bytesRead: [189544]      outStream.Length [189544]
bytesRead: [0]           outStream.Length [189544]
total bytes read [189544]
Unompressed stream size 189544

Also note the 3rd read of buffersize 65536 ex: bytesRead: [58472] Clearly this should also be 65536 as there still data left on the buffer?

Any ideas will be greatly appreciated.

tia

• Jaco

Answer 1

My psychic powers tells me that you do in fact have a working decompression implementation, but have forgotten to flush the compression stream before.

Answer 2

You should always call Close() on compression streams. Please note that Flush() is not enough. I suspect that because of this the deflate stream is missing data.

Answer 3

Well, I couldn't spot your problem, but follow some code I wrote sometime ago for ICSharpCode.SharpZipLib;

byte[] compressedData;
using(MemoryStream ms = new MemoryStream())
{
    Deflater deflater = new Deflater(Deflater.BEST_COMPRESSION, true);
    Stream s = new DeflaterOutputStream(ms, deflater);
    s.Write(sendData, 0, sendData.Length);
    s.Close();
    compressedData = (byte[])ms.ToArray();
}

// ...

MemoryStream inflated = new MemoryStream();
using (Stream inflater = new InflaterInputStream(
    inputStream, new Inflater(true)))
{
    int count = 0;
    byte[] deflated = new byte[4096];
    while ((count = inflater.Read(deflated, 0, deflated.Length)) != 0)
    {
        inflated.Write(deflated, 0, count);
    }
    inflated.Seek(0, SeekOrigin.Begin);
}
byte[] content = new byte[inflated.Length];
inflated.Read(content, 0, content.Length);