Handling arbitrary bit length data in Delphi?

Question

I am working on a display/control utility to replace an ancient dedicated hardware controller for a piece of industrial machinary. The controller itself is beyond repair (someone replaced the 1 amp fuse with a 13 amp one "because it kept blowing"). The hardware interface is through a standard RS232 port. The data format is dedicated:

No control characters are used with the exeption of ETB (Chr 23) to demark end of a message.

The data is 7-bit, but only a subset of the possible 7-bit characters is used. The content of each 7-bit data character is therefore effectively reduced to only 6 bits of data.

The data is not character aligned, e.g. for the first message type, the first 3 bits are the message type, the next 8 bits are a counter, the next 15 bits are a data value, next 7 bits are a value etc

So reducing the data from it's 7-bit carrier to it's 6 bit content gives (for example)

|  6 ||  6 ||  6 ||  6 ||  6 ||  6 ||  6 ||~ 
001001010001010100101010101101010101110111 ~
|3||  8   ||     15       ||  7   || ~~   
 M    C          D            D        D
 s    o          a            a        a 
 g    u          t            t        t 
      n          a            a        a
      t

Specific messages are fixed length but the different messages are of different lengths and contain different numbers of parameters.

I have a working prototype handling one specific message type, but it is currently using way too many case statements ;-).

I am looking for suggestions as to a clean way of handling such packed, arbitrary bit length data?

Answer 1

One possible way to handle this, while being terribly inefficient in terms of memory usage, would be to break up the bits as the data is read in. So, let's say you read in the data from the port in 8-bit (1-byte) chunks. Your first read would bring in 00100101. Break this into an array of 8 integers (e.g. bits[0] := 0; bits[1] := 0; bits[2] := 1; ...)

Now you can write helper routine(s) that will retrieve the value you are looking for from the array:

function getInt(start, len: integer): integer;
function getChar(start: integer): String;

These functions would use the start (and possibly len) parameters to combine the appropriate bits out of your array into a usable value.

Answer 2

Use SHL/SHR along with masking to read out of your buffer. I would write a few functions to operate against the buffer (which I would declare as an array of byte) and return the value of a specific number of bits form a starting bit position. For instance, lets say that your largest value will never be more than 16 bits (a word). Since your mapped to an array of bytes, if you always grab 3 bytes from the array (watch for the upper bounds) and throw into an integer you can then mask and shift to get your specific value. The location of the bytes you want to get will then be (assuming a 0 based array):

Byte1Index = FirstBit DIV 8;
Byte2Index = Byte1Index + 1;
Byte3Index = Byte1Index + 2;

Pull these into your integer from your array.

TempInt := 0;
if Byte1Index <= High(Buffer) then
  TempInt := Buffer[Byte1Index];
if Byte2Index <= High(Buffer) then 
  TempInt := TempInt OR ((Buffer[Byte2Index] and $000000FF) SHL 8);
if Byte3Index <= High(Buffer) then
  TempInt := TempInt OR ((Buffer[Byte2Index] and $000000FF) SHL 16);

Then adjust your result to align properly

if FirstBit MOD 8 <> 0 then
  TempInt := TempInt SHR (FirstBit MOD 8);

Then mask against the most number of bits you want to return:

Result := TempInt AND $00003FFF;

You can build the final mask programatically:

FinalMask := ($FFFFFFFF shl bitcount) xor $FFFFFFFF;

EDIT

This method is currently limited to 32 bits, but can be extended to at the most 64 bits by changing the masks from 32bit integers to 64bit integers (from $FFFFFFFF to $FFFFFFFFFFFFFFFF for example). Performance gains can also be made by not loading the extra bytes if they aren't needed, I just included here examples for 16 bits, however you will always want to grab at least an extra byte if the Bitsneeded + FirstBiT MOD 8 <> 0

For the messaging side of things I would create a base object which knows how to read data out of a buffer, then extend this into an object which knows how to also read parameters, and then create object descendants which know how to process each message type. You would still have a case statement, but it would be at a simple dispatcher level doing nothing more than passing the buffer off to the appropriate object to handle.

Answer 3

I'd probably use ASM blocks to handle this with some assembler code - if you can use x86 assembly it would be much easier to parse the data and convert them to a more readable format to pass along.