How to determine compressed size from zlib for gzipped data?

Question

I'm using zlib to perform gzip compression. zlib writes the data directly to an open TCP socket after compressing it.

/* socket_fd is a file descriptor for an open TCP socket */
gzFile gzf = gzdopen(socket_fd, "wb");
int uncompressed_bytes_consumed = gzwrite(gzf, buffer, 1024);

(of course all error handling is removed)

The question is: how do you determine how many bytes were written to the socket? All the gz* functions in zlib deal with byte counts/offsets in the uncompressed domain, and tell (seek) doesn't work for sockets.

The zlib.h header says "This library can optionally read and write gzip streams in memory as well." Writing to a buffer would work (then I can write the buffer to the socket subsequently), but I can't see how to do that with the interface.

Answer 1

You'll be able to do this with the deflate* series of calls. I'm not going to show you everything, but this example program (which I had named "test.c" in my directory) should help you get started:

#include <zlib.h>
#include <stdlib.h>
#include <stdio.h>

char InputBufferA[4096];
char OutputBufferA[4096];

int main(int argc, char *argv[])
{
    z_stream Stream;
    int InputSize;
    FILE *FileP;

    Stream.zalloc = malloc;
    Stream.zfree = free;
    /* initialize compression */
    deflateInit(&Stream, 3);
    FileP = fopen("test.c", "rb");
    InputSize = fread((void *) InputBufferA, 1, sizeof(InputBufferA), FileP);
    fclose(FileP);
    Stream.next_in = InputBufferA;
    Stream.avail_in = InputSize;
    Stream.next_out = OutputBufferA;
    Stream.avail_out = sizeof(OutputBufferA);
    deflate(&Stream, Z_SYNC_FLUSH);
    /* OutputBufferA is now filled in with the compressed data. */
    printf("%d bytes input compressed to %d bytes\n", Stream.total_in, Stream.total_out);
    exit(0);
}

Consult the deflate documentation from zlib.h.

Answer 2

zlib can, in fact, write gzip formatted data to a buffer in memory.

This zlib faq entry defers to comments in zlib.h. In the header file, the comment for deflateInit2() mentions that you should (arbitrarily?) add 16 to the 4th parameter (windowBits) in order to cause the library to format the deflate stream with the gzip format (instead of the default "zlib" format).

This code gets the zlib state set up properly to encode gzip to a buffer:

#include <zlib.h>
z_stream stream;
stream.zalloc = Z_NULL;
stream.zfree = Z_NULL;
stream.opaque = Z_NULL;
int level = Z_DEFAULT_COMPRESSION;
int method = Z_DEFLATED;  /* mandatory */
int windowBits = 15 + 16; /* 15 is default as if deflateInit */
                          /* were used, add 16 to enable gzip format */
int memLevel = 8;         /* default */
int strategy = Z_DEFAULT_STRATEGY;
if(deflateInit2(&stream, level, method, windowBits, memLevel, strategy) != Z_OK)
{
    fprintf(stderr, "deflateInit failed\n");
    exit(EXIT_FAILURE);
}

/* now use the deflate function as usual to gzip compress */
/* from one buffer to another. */

I confirmed that this procedure yields the exact same binary output as the gzopen/gzwrite/gzclose interface.