What is best compression scheme for very small data as1.66kBytes

views:

answers:

+3 Q:

What is best compression scheme for very small data as1.66kBytes

this data is stored in an array (using C++) and is repitition of 125 bits each one varying from other also has 8 messages of 12 ASCII characters each at end..... Please suggest that should i use diffential compression within array and how? or i should apply some other compression scheme as whole onto the array?

thanks in advance

Regards

Haider

+1 A:

Generally you can compress data that has some sort of predictability or redundancy. Dictionary based compression (e.g. ZIP style algorithms) traditionally don't work well on small chunks of data because of the need to share the selected dictionary.

In the past, when I have compressed very small chunks of data with somewhat predictable patterns, I have used SharpZipLib with a custom dictionary. Rather than embed the dictionary in the actual data, I hard-coded the dictionary in every program that needs to (de)compress the data. SharpZipLib gives you both options: custom dictionary, and keep dictionary separate from the data.

Again this will only work well if you can predict some patterns to your data ahead of time so that you can create an appropriate compression dictionary, and it's feasible for the dictionary itself to be separate from the compressed data.

Eric J. 2010-05-23 20:51:03

Well Eric.J thanks for your answeractually i need to transmit the compressed data every 4 seconds and actually the 125 bits data isnt much prbabilistic

Mujtaba Haider 2010-05-23 21:21:55

@Mujtaba: What kind of data is represented in the 125 bits?

Eric J. 2010-05-23 21:30:31

its aircraft information e.g latitude,longitude ,speed,altitude,heading,id,type etc..

Mujtaba Haider 2010-05-24 05:15:15

@Mujtaba: That kind of data is actually **highly** compressible if you're currently sending it as text. There are mostly digits and the decimal symbol. Even if binary, you should still get good compression. Headings are only 0-359, airspeed and altitude fall in narrow ranges as does lat/long. I suspect SharpZipLib with custom dictionary would work well. Lempel-Ziv probably would not as there's not a lot of data in a given stream.

Eric J. 2010-05-24 15:36:24

@Eric right now i am generating this data randomly and storing in an array.This array is then transmitted after appending header and trailer through RS-232 in lab model but actual tx will be over air in real time. Also i wanted to ask you that that as array carrying the radar data has 100 aircraft tracks dataplus 8 messages of 12 ascii ch each,will compression achieved by tranmitting inter track difference be more effective than applying compression as whole on the array?

Mujtaba Haider 2010-06-02 08:29:52

A difference-based compression should be much more effective since the aircraft will tend to be the same (only occasionally will one be added or drop off) and the coordinates will be very similar. However, if you lose one packet of data with a difference strategy, difference compression breaks down (what if you lose the packet that indicates a new aircraft is in the sector...). Having a "key" frame that contains full data every few seconds would solve that.

Eric J. 2010-06-02 15:45:44

@Eric J.Now i am implementing differential compression on two consecutive arrays containing 13268 bits each and i assume that the resultant array would be much more feasible for LZ compression any suggestions??

Mujtaba Haider 2010-06-11 05:23:48

Norman Ramsey 2010-05-23 21:36:30

Thanks RAMSEY do these bzip2 and LZ techniques work for array data?

Mujtaba Haider 2010-05-24 06:28:16

@Mujtaba absolutely yes: any sequence of bytes in memory.

Norman Ramsey 2010-05-25 01:19:11

would'nt applying these compression techniques make the bit stream longer because these some over heads for makingcompression possible?

Mujtaba Haider 2010-06-02 08:52:22

@Mujtaba it depends what's in the bit streams. Bit streams with redundancy get shorter. The total overhead for compression can easily be reduced to one bit, however, so it shouldn't be a concern. Whether standard techniques give you actual compression is an experimental question.

Norman Ramsey 2010-06-02 14:08:00

@Norman Now i am implementing differential compression on two consecutive arrays containing 13268 bits each and i assume that the resultant array would be much more feasible for LZ compression any suggestions?? –

Mujtaba Haider 2010-06-11 05:25:47

@Norman Ramsey ????

Mujtaba Haider 2010-06-17 05:39:59

ansaurus

tags:

views:

answers:

What is best compression scheme for very small data as1.66kBytes

related questions