Using LINQ, need help splitting a byte array on data received from Silverlight sockets

Question

The message packats received contains multiple messages deliniated by a header=0xFD and a footer=0xFE

// sample message packet with three
// different size messages
List<byte> receiveBuffer = new List<byte>();
receiveBuffer.AddRange(new byte[]
  { 0xFD, 1, 2, 0xFE, 
    0xFD, 1, 2, 3, 4, 5, 6, 7, 8, 0xFE,
    0xFD, 33, 65, 25, 44, 0xFE});


// note: this sample code is without synchronization, 
//       statements, error handling...etc.
while (receiveBuffer.Count > 0)
{
    var bytesInRange = receiveBuffer.TakeWhile(n => n != 0xFE);

    foreach (var n in bytesInRange)
        Console.WriteLine(n);

    // process message.. 
    // 1) remove bytes read from receive buffer
    // 2) construct message object...
    // 3) etc...

     receiveBuffer.RemoveRange(0, bytesInRange.Count());

}

As you can see, (including header/footer) the first message in this message packet contains 4 bytes, and the 2nd message contains 10 bytes,a and the 3rd message contains 6 bytes.

In the while loop, I was expecting the TakeWhile to add the bytes that did not equal the footer part of the message.

Note: Since I am removing the bytes after reading them, the header can always be expected to be at position '0'.

I searched examples for splitting byte arrays, but non demonstrated splitting on arrays of unknown and fluctuating sizes.

Any help will be greatly appreciated. thanks much!

Answer 1

the idea is:

while not the end of receiveBuffer 
 if receiverbuffer[actualposition] != 0xfe
    insert this position in a listA
  if receiverbuffer[actualposition] == 0xfe
    insert the listA into another listB
    listA become null and you go to next line
go to next position of receivebuffer

so at end of process, you'll have a list>

i hope it doesn't look like much confusion

Answer 2

Not sure if this is just a bug introduced in your demonstration code but you do need to add one to the count when removing the previous message from the buffer:

receiveBuffer.RemoveRange(0, bytesInRange.Count() + 1);

instead of

receiveBuffer.RemoveRange(0, bytesInRange.Count());

With this one change, the code printed out every byte except the end marker of each of the three messages.

The following variation on your code prints every body byte for each message:

List<byte> receiveBuffer = new List<byte>();
receiveBuffer.AddRange(new byte[] 
{
    0xFD, 1, 2, 0xFE,  
    0xFD, 1, 2, 3, 4, 5, 6, 7, 8, 0xFE, 
    0xFD, 33, 65, 25, 44, 0xFE
});

while (receiveBuffer.Count > 0)
{
    var bytesInRange = receiveBuffer.Skip(1).TakeWhile(n => n != 0xFE);

    foreach (var n in bytesInRange)
        Console.Write("{0} ", n);

    Console.WriteLine("\n");
    receiveBuffer.RemoveRange(0, bytesInRange.Count() + 2);
}

Answer 3

The really tricky part of this is that sockets are a stream of data, so your buffer may in fact contain only a partial message. I have code here implementing a single-byte delimiter-based "framer" that correctly handles partial reads. It's fully unit tested.

Note the following design tips for "framer" classes, from decades of experience:

• Separate your message buffering into a class. The buffering code is complex enough without dealing with the asynchronous socket stuff. The socket event handlers can be responsible for keeping asynchronous reads going all the time, handling 0-length reads, and error handling. Then they should pass off the data to the buffering class, which is responsible for doing the actual framing.
• When writing the message buffering class, you end up with cleaner code if you change your way of thinking about the data. Instead of a chunk of data arriving at a socket and being "pushed" through the buffering class, think of it as the buffering class issuing an implicit "read request"; when a chunk of data comes in, do a loop satisfying the current "read request" until the chunk is all used up.

Answer 4

Since you already have proper framing, is there some reason Daniel's solution isn't working for you?

If you want something just using LINQ, this can be done:

int messageIndex = 0;
var test = receiveBuffer
    // Remove SOT bytes
    .Where(x => x != 0xFD)
    // Assign each byte as being part of a message, indexing on EOT
    .Select(x =>
        {
            if (x == 0xFE) ++messageIndex;
            return new { Byte = x, MessageIndex = (x == 0xFE ? -1 : messageIndex) };
        })
    // Remove EOT bytes
    .Where(x => x.MessageIndex != -1)
    // Group by message index
    .GroupBy(x => x.MessageIndex)
    // Strip message index and convert the bytes in each message to a List<byte>
    .Select(x => x.Select(y => y.Byte).ToList())
    // Execute the query, saving in a List<List<byte>>
    .ToList();

However, I really feel Daniel's solution is more readable and maintainable. Beware the dark side.

If you insist on LINQ, I'd recommend writing a Partition extension method that cleans all this up, so you could have code like receiveBuffer.Where(x => x != 0xFD).Partition(0xFE). No such function currently exists.