views:

133

answers:

2

I want to put some compressed data into a remote repository.
To put data on this repository I can only use a method that take the name of the resource and its content as a String. (like data.txt + "hello world").
The repository is moking a filesystem but is not, so I can not use File directly.

I want to be able to do the following:

  1. client send to server a file 'data.txt'
  2. server compress 'data.txt' into a compressed file 'data.zip'
  3. server send a string representation of data.zip to the repository
  4. repository store data.zip
  5. client download from repository data.zip and his able to open it with its favorite zip tool

The problem arise at step 3 when I try to get a string representation of my compressed file.

Here is a sample class, using the zip*stream and that emulate the repository showcasing my problem.
The created zip file is working, but after its 'serialization' it's get corrupted.
(the sample class use jakarta commons.io )

Many thanks for your help.

package zip;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import java.util.zip.ZipOutputStream;

import org.apache.commons.io.FileUtils;

/**
 * Date: May 19, 2010 - 6:13:07 PM
 *
 * @author Guillaume AME.
 */
public class ZipMe {
    public static void addOrUpdate(File zipFile, File ... files) throws IOException {

        File tempFile = File.createTempFile(zipFile.getName(), null);
        // delete it, otherwise you cannot rename your existing zip to it.
        tempFile.delete();

        boolean renameOk = zipFile.renameTo(tempFile);
        if (!renameOk) {
            throw new RuntimeException("could not rename the file " + zipFile.getAbsolutePath() + " to " + tempFile.getAbsolutePath());
        }
        byte[] buf = new byte[1024];

        ZipInputStream zin = new ZipInputStream(new FileInputStream(tempFile));
        ZipOutputStream out = new ZipOutputStream(new FileOutputStream(zipFile));

        ZipEntry entry = zin.getNextEntry();
        while (entry != null) {
            String name = entry.getName();
            boolean notInFiles = true;
            for (File f : files) {
                if (f.getName().equals(name)) {
                    notInFiles = false;
                    break;
                }
            }
            if (notInFiles) {
                // Add ZIP entry to output stream.
                out.putNextEntry(new ZipEntry(name));
                // Transfer bytes from the ZIP file to the output file
                int len;
                while ((len = zin.read(buf)) > 0) {
                    out.write(buf, 0, len);
                }
            }
            entry = zin.getNextEntry();
        }
        // Close the streams
        zin.close();
        // Compress the files
        if (files != null) {
            for (File file : files) {
                InputStream in = new FileInputStream(file);
                // Add ZIP entry to output stream.
                out.putNextEntry(new ZipEntry(file.getName()));
                // Transfer bytes from the file to the ZIP file
                int len;
                while ((len = in.read(buf)) > 0) {
                    out.write(buf, 0, len);
                }
                // Complete the entry
                out.closeEntry();
                in.close();
            }
            // Complete the ZIP file
        }
        tempFile.delete();
        out.close();

    }

    public static void main(String[] args) throws IOException {

        final String zipArchivePath = "c:/temp/archive.zip";
        final String tempFilePath = "c:/temp/data.txt";
        final String resultZipFile = "c:/temp/resultingArchive.zip";

        File zipArchive = new File(zipArchivePath);
        FileUtils.touch(zipArchive);

        File tempFile = new File(tempFilePath);
        FileUtils.writeStringToFile(tempFile, "hello world");
        addOrUpdate(zipArchive, tempFile);

        //archive.zip exists and contains a compressed data.txt that can be read using winrar

        //now simulate writing of the zip into a in memory cache
        String archiveText = FileUtils.readFileToString(zipArchive);
        FileUtils.writeStringToFile(new File(resultZipFile), archiveText);

        //resultingArchive.zip exists, contains a compressed data.txt, but it can not
        //be read using winrar: CRC failed in data.txt. The file is corrupt

    }

}
+2  A: 

Zip files are binary. String handling in Java is textual and might be mangling what it sees as CRLFs, zero bytes and EOF markers. When it comes to reading and rewriting the zipfile, I suggest you try with readFileToByteArray and writeByteArrayToFile as an experiment. If that works then I'd suspect the String handling is to blame.

crazyscot
I concur. When passing binary data, you must use Byte, not Char or String, which are UTF16.
Marcus Adams
@Marcus You mean UTF-8 (check sys props).In order for this work you will need to load the entire file's contents as a byte array before converting it to a string. Whenever you encode/decode bytes to/from strings there's a danger that characters (can cross multiple-bytes) can be incorrectly encoded/decoded if a buffer is used.
Raymond
@Raymond, thanks. I probably wasn't clear. String and Char are UTF16 (multi-byte). Bytes are, obviously, single byte, and that's what you want.
Marcus Adams
Thank you all for your precious help. I still have a problem. I guess it came from the encoding. But I can't figure how to solve it. I need a string representation of the byte arrays, and I don't know how to get it. I've try with many charset while encoding/decoding the string but no one is working.
Guillaume
Base64 encoding ought to do the trick. http://iharder.sourceforge.net/current/java/base64/
crazyscot
+1  A: 

server send a string representation of data.zip to the repository

So you want to get a string (i.e. textual) representation of a zip (i.e. binary) stream.

Base64 is the most popular way to do this.

One popular Java implementation is from Apache commons (codec component)

leonbloy