fastest c++ file compression library available?

Question

I have a need to find the best library to compress in memory data. I am currently using zlib but I am wondering if there is a better compression library; better in terms of performance and memory footprint.

It should be able to handle multiple files in the same archive.

I am looking for a C/C++ library.

Performance is the key factor. The files that are being compressed are small to large XML files.

Answer 1

Most libs have a C API rather than C++. You can look at lzo, which is fairly fast but not as good in terms of compression ratio.

http://www.oberhumer.com/opensource/lzo/

Compression tools shoot-out: http://compression.ca/act/act-summary.html

EDIT: More details about lzo (from its website, thanks Chris for correcting the link)

• Requires no memory for decompression.
• Compression is pretty fast.
• Requires 64 kB of memory for compression.

Note that lzo works on memory buffers, so you'll have to handle the conversion to/from files yourself.

Also note that lzo is GPL which may be a problem, depending on your application.

Answer 2

Here is a comparison table and here is another one, and I belive some of the tools have SDKs available...

Answer 3

Here is a succintly written Linux Journal article from 2005 which seems to declare LZMA the winner for transferring files over a network. The results are based on percentage increase in 'effective bandwidth'.

If you need a small, fast compression library to increase the effective bandwidth of an application, then you should check out LZMA.

You'll see LZMA beating out both zlib and LZO... that said, last time I checked (2-3 years ago) the SDK was crap because it was literally extracted from 7zip and wasn't designed as a library... and for projects that required compression I used zlib because of the various client-friendly APIs available.

Answer 4

Look at qlz lib ( QuickLZ ) it provides on GPL license , and it is open source

Answer 5

In all reality, you have to decide what you want. Speed, or compression ratio. You can't have both...

In my findings, zlib is probably the best out there in terms of memory foot prnt, and speed, along with a decent compression ratio. There ARE faster ones out there, but how much compression do you need? LZ77/LZMA is probably right up there with deflate as an algorithm, however, I'm not sure of it's memory usage. I know it can and will compress files better than deflate...

There are tons of ways to handle this, so I'd figure out exactly what you have to accompolish first, THEN pick an algorithm to go with. And since you already mentioned you are compressing XML, I'd stick to algorithm data that shows text operations, and not focus so much on binary or any other data being compressed.

Answer 6

http://www.oberhumer.com/opensource/lzo/ is faster (twice as fast i believe) than zlib and compresses 10% worse

Answer 7

The best way is looking at a benchmark site, that display the cpu time without i/o. Ex. compressionratings

Answer 8

try QuickLZ www.quicklz.com

Answer 9

Checkout freearc.org . It is very fast but at the same time gives a higher compression ratio than RAR/7z/ZIP/GZ etc.YOu can directly use the source code for programming.

Answer 10

zlib and gzip, while not being the fastest to decompress, are pretty good overall and they have usually good GUI tools.

• Boost Iostreams can handle both pretty simply.

• Minizip (or Info-ZIP) can handle ZIP (archived lzip files)

I don't know much about those two libraries performance comparison but the underlying library is zlip for ZIP archive handling.