compressing and decompressing source data gives result different than source data

Question

Hello!

In my app I need to Decompress data written by DataContractSerializer to compression Deflate Stream in another app, edit the decompressed data and Compress it again.

Decompression works fine, but not for data compressed by me.

The problem is that when I do this: byte[] result = Compressor.Compress(Compressor.Decompress(sourceData));

the length of the result byte array is different than sourceData array.

For example:

    string source = "test value";
    byte[] oryg = Encoding.Default.GetBytes(source);

    byte[] comp = Compressor.Compress(oryg);
    byte[] result1 = Compressor.Decompress(comp);

    string result2 = Encoding.Default.GetString(res);

and here result1.Length is 0 and result2 is "" of course

Here is the code of my Compressor class.

public static class Compressor
{
    public static byte[] Decompress(byte[] data)
    {
        byte[] result;

        using (MemoryStream baseStream = new MemoryStream(data))
        {
            using (DeflateStream stream = new DeflateStream(baseStream, CompressionMode.Decompress))
            {
                result = ReadFully(stream, -1);
            }
        }

        return result;
    }

    public static byte[] Compress(byte[] data)
    {
        byte[] result;

        using (MemoryStream baseStream = new MemoryStream())
        {
            using (DeflateStream stream = new DeflateStream(baseStream, CompressionMode.Compress, true))
            {
                stream.Write(data, 0, data.Length);
                result = baseStream.ToArray();
            }
        }

        return result;
    }

    /// <summary>
    /// Reads data from a stream until the end is reached. The
    /// data is returned as a byte array. An IOException is
    /// thrown if any of the underlying IO calls fail.
    /// </summary>
    /// <param name="stream">The stream to read data from</param>
    /// <param name="initialLength">The initial buffer length</param>
    private static byte[] ReadFully(Stream stream, int initialLength)
    {
        // If we've been passed an unhelpful initial length, just
        // use 32K.
        if (initialLength < 1)
        {
            initialLength = 65768 / 2;
        }

        byte[] buffer = new byte[initialLength];
        int read = 0;

        int chunk;
        while ((chunk = stream.Read(buffer, read, buffer.Length - read)) > 0)
        {
            read += chunk;

            // If we've reached the end of our buffer, check to see if there's
            // any more information
            if (read == buffer.Length)
            {
                int nextByte = stream.ReadByte();

                // End of stream? If so, we're done
                if (nextByte == -1)
                {
                    return buffer;
                }

                // Nope. Resize the buffer, put in the byte we've just
                // read, and continue
                byte[] newBuffer = new byte[buffer.Length * 2];
                Array.Copy(buffer, newBuffer, buffer.Length);
                newBuffer[read] = (byte)nextByte;
                buffer = newBuffer;
                read++;
            }
        }
        // Buffer is now too big. Shrink it.
        byte[] ret = new byte[read];
        Array.Copy(buffer, ret, read);
        return ret;
    }
}

Please help me with this case if You can. Best regards, Adam

Answer 1

(edited: switched from using flush, which still might not flush out all bytes, to now ensuring deflate is disposed first, as per Phil's answer here: http://stackoverflow.com/questions/2118904/zip-and-unzip-string-with-deflate)

Before attempting to read from backing store, you have to ensure the deflate stream has fully flushed itself when compressing, allowing deflate to finish compressing and write final bytes. Closing the deflate steam, or disposing of it, will achieve this.

public static byte[] Compress(byte[] data)
{
    byte[] result;

    using (MemoryStream baseStream = new MemoryStream())
    {
        using (DeflateStream stream = new DeflateStream(baseStream, CompressionMode.Compress, true))
        {
            stream.Write(data, 0, data.Length);
        }
        result = baseStream.ToArray();  // only safe to read after deflate closed
    }

    return result;
}    

Also your ReadFully routine looks incredibly complicated and likely to have bugs. One being:

while ((chunk = stream.Read(buffer, read, buffer.Length - read)) > 0)

When reading the 2nd chunk, read will be greater than the length of the buffer, meaning it'll always pass a negative value to stream.Read for the number of bytes to read. My guess is that it'll never read the 2nd chunk, returning zero, and fall out of the while loop.

I recommend Jon's version of ReadFully for this purpose: http://stackoverflow.com/questions/221925/creating-a-byte-array-from-a-stream