Why is my .zip file corrupted after an HTTP file upload?

Question

I'm trying to use a CGI script to accept and save a file from a program that is using an HTTP POST to send a zip file.

In the MIME section of the HTTP header it looks something like this:

Content-Disposition: form-data; name="el_upload_file_0"; filename="BugReport.zip";\r\n
Content-Type: application/octet-stream\r\n\r\n

In my CGI code I'm using this:

 use CGI;
 use strict;
 my $cgi = CGI->new;
 my $upload_file = $cgi->upload('el_upload_file_0');
 my $time = time;

 my $filename = "/tmp/$time.zip";
 open TMP, ">$filename";
 binmode TMP;
 while (<$upload_file>) {
    print TMP $_;
 }
 close TMP;

The file that keeps getting saved is somehow getting corrupt and is not a valid zip file. The HTTP request is being sent by a C# app and it's possible that it might be sending a corrupt zip file, but I doubt it. Is there anything I can do to troubleshoot further?

Answer 1

What's the difference between the file you uploaded and the file that you ended up saving? Look at a hex dump of each.

I know it sounds stupid, but trying merely copying the file you are trying to upload to the server without using the CGI script. Can you still unzip it there? Likewise, can you take the uploaded file from the server, copy it back to your client machine, and unzip it there?

What's the rest of the HTTP header look like? Are you changing character sets or anything?

I don't suspect a problem with file translations since CGI should already set that for you. So, once anyone says "should", you have to double check :). There's a line in CGI.pm that auto-detects systems that need binmode, so I don't think you need that on $upload_file. However, maybe CGI.pm gets it wrong for you:

 $needs_binmode = $OS=~/^(WINDOWS|DOS|OS2|MSWin|CYGWIN|NETWARE)/;

You might try setting the variable to true yourself just to make sure:

 use CGI;
 $CGI::needs_binmode = 1;

Answer 2

You're reading in a .zip file line by line, which is a big mistake. Lines are only relevant for text files, after all.

Read the whole thing in one shot, or if you must, do it in reasonably sized chunks. In this example it's being read in 1024 byte chunks, but you can easily use a much larger value, like 16MB (1 << 24) or whatever seems appropriate:

my $data;
while (read($upload_file, $data, 1024))
{
  print TMP, $data;
}

Answer 3

The "\r\n"'s in the header might be a clue. Does the output file contain "\r\n" sequences? Can you/should you do a binmode on the $upload_file filehandle?