compressing socket send data

Question

Hi,

I'm trying to send a lot of data(basically data records converted to a string) over a socket and its slowing down the performance of the rest of my program. Is it possible to compress the data using gzip etc. and uncompress it at the other end?

Answer 1

Yes, but compression and decompression have their costs as well.

You might want to consider using another process or thread to handle the data transfer; this is probably harder than merely compressing, but will scale better when your data load increases n-fold.

Answer 2

Of course you can do that. When sending binary data, you have to take care of endiannes of the platform.

However, are you sure your performance problems will be solved through compression of sent data? You'll still have additional steps (compression/decompression, possibly solving endiannes issues).

Think about how the communication through sockets is done. Are you using synchronous or asynchronous communication. If you do the reads and writes synchronous, then you can feel performance penalities...

Answer 3

Yes, it's possible. zlib is one library for doing this sort of compression and decompression. However, you may be better served by serializing your data records in a binary format rather than as a string; that should improve performance, possibly even more so than using compression.

Answer 4

Yes. The easiest way to implement this is to use the venerable zlib library.

The compress() and uncompress() utility functions may be what you're after.

Answer 5

You may use AdOC a library to transparently overload socket system calls

http://www.labri.fr/perso/ejeannot/adoc/adoc.html

It does compression on the fly if it finds that it would be profitable.