How to solve Gzip Magic Number Missing

Question

I have a string that I Gzip on the server and download to a client using the WebClient class. When I try to uncompress it, I get the error message that the Magic Number is missing. I have tried both the GZipStream class and the ICSharpLib methods of solving this, so I'm at a loss.

The compression/decompression works if I omit the step of downloading via the WebClient (using DownloadData which returns the data as byte[]), so I can only assume that there is some problem with the data getting truncated or corrupted some how, but since it's compressed data, I'm not sure how to debug this.

Here's the code snippet that seems to be the offending portion:

   byte[] response
   try {
        response = client.DownloadData(Constants.GetSetting("SyncServer"));
   } catch {
        MessageBox.Show("There was a problem synchronizing the data. Please try verify the supplied credentials or try again later.", "Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
        return;
   }

   int rows = SQLiteAPI.ImportStatHistoryXML(CurrentUser.User, myCampus, Convert.ToBase64String(response));

public static int ImportStatHistoryXML(Person tempPerson, Campus tempCampus, string xmlFile) {
      byte[] encryptedFile = Convert.FromBase64String(xmlFile);
      MemoryStream memStream = new MemoryStream(encryptedFile);
      memStream.ReadByte();
      GZipInputStream stream = new GZipInputStream(memStream);
      MemoryStream memory = new MemoryStream();
      byte[] writeData = new byte[4096];
      int size;

      while (true) {
       size = stream.Read(writeData, 0, writeData.Length);
       if (size > 0) {
        memory.Write(writeData, 0, size);
       } else {
        break;
       }
      }
      stream.Close();
      memory.Position = 0;
      StreamReader sr = new StreamReader(memory);
      string decompressed = sr.ReadToEnd();
      DataSet tempSet = new DataSet();
      StringReader xmlReader = new StringReader(decompressed);
      tempSet.ReadXml(xmlReader);
      DataTable statTable = tempSet.Tables["Stats"];
...more unrelated processing of the table
}

Any help would be appreciated. P.S. I'm using the Base64 string to be able to pass back and forth across the web. This may in fact be the area I am messing up in since I've not done web requests and responses between a desktop app and a web service before.

Answer 1

First, I don't think the snippet is valid, because DownloadString returns (as expected) a String.

Now, do I understand right that it works correctly when you use DownloadData and incorrectly when you use DownloadString? That makes sense because it is not valid to decode Gzip data as Unicode.

EDIT:

Okay, the ToBase64String and FromBase64String should be okay. But if you can avoid it and pass the byte[] directly, that would be good.

public static int ImportStatHistoryXML(Person tempPerson, Campus tempCampus, byte[] compressedFile) {

Then you would get rid of the first line of the function (the decode from base64). Note we're renaming encryptedFile to compressedFile.

The line:

memStream.ReadByte();

should not be there. You are reading a byte and discarding it. If everything is as we expect that byte is 0x1F, part of the gzip magic number.

Then, I think you're using the wrong gzip class. You want GZipStream. It is constructed like:

GZipStream stream = new GZipStream(memStream, CompressionMode.Decompress);

Then, you use StreamReader directly on that:

StreamReader sr = new StreamReader(stream);

It would help if you knew the encoding, but hopefully the default will be correct. Then it seems correct from there. So, overall we get the below.

public static int ImportStatHistoryXML(Person tempPerson, Campus tempCampus, byte[] compressedFile) {
    MemoryStream memStream = new MemoryStream(compressedFile);
    GZipStream gzStream = new GZipStream(memStream, CompressionMode.Decompress);
    StreamReader sr = new StreamReader(gzStream);
    string decompressed = sr.ReadToEnd();
    DataSet tempSet = new DataSet();
    StringReader xmlReader = new StringReader(decompressed);
    tempSet.ReadXml(xmlReader);
    DataTable statTable = tempSet.Tables["Stats"];

    //...
}