How to process compressed data in Java

Question

I have some data which takes up more than 50MB in an uncompressed file, but compresses down to less than half a MB using gzip.

Most of this is numerical data. I'm trying to figure out how to process this data without having to uncompress it completely. For example, if this data contains a couple of strings and 5 or so numerical values per record, is there a way I can uncompress a single row (or a small set of rows), process them, then discard them?

Unix provides utilities such as zcat, grep, etc. which operate directly on compressed data, I'd like to do the same in Java.

Thanks

Answer 1

Just wrap your FileInputStream in a GZipInputStream:

public static BufferedReader createReader (File f, String encoding) throws IOException
{
    try
    {
        InputStream in = new FileInputStream (f);
        if (f.getName ().endsWith (".gz"))
            in = new GZIPInputStream (in, 10240);

        return new BufferedReader (new InputStreamReader (in, encoding));
    }
    catch (UnsupportedEncodingException e)
    {
        throw new RuntimeException("Missing encoding "+encoding, e);
    }
}