LZO Decompression Buffer Size

Question

I am using MiniLZO on a project for some really simple compression tasks. I am compressing with one program, and decompressing with another. I'd like to know how much space to allocate for the decompression buffer. I am fine with over-allocating space, if it can save me the trouble of having to annotate my output file with an integer declaring how much space the decompressed data should take. How would I figure out how much space it could possibly take?

After some consideration, I think this question boils down to the following: What is the maximum compression ratio of lzo1x compression?

Answer 1

The max size of the decompressed data is clearly the same as the max size of the data you compressed in the first place.

If there is an upper bound on your input size then I guess you can use it, but I have to say the usual way of doing this is to add a header to your compressed buffer which specifies the uncompressed size.

Answer 2

Since you control both the compressor and the decompressor, I suggest you compress the input in fixed-sized blocks. In my application I compress up to 64KB in each block, then emit the size of the compressed block and the compressed data itself, so the compressed stream actually looks like a series of compressed blocks:

length_of_block_1
block_1
length_of_block_2
block_2
...

The decompressor just reads each compressed block and decompresses it into a 64KB buffer, since I know the block was produced by compressing a 64KB block.

Hope that helps,

Eric Melski