How can I compound byte[] buffers into a List<byte>?

Question

So I'm receiving data over a socket using a buffer (byte[]) of size 1024, and I want to combine the reads together to form the entire packet in the event that they're bigger than 1024 bytes. I chose a List to store the entire packet, and what I want to do is add each buffer read to it as it comes in. I'd want to do:

List.AddRange(Buffer);

But in the event that the buffer isn't full a bunch of empty bytes would get padded to the end. So naturally what I would want to do is add only a certain range of bytes to the List, but there is no such method. I could always create a temporary byte array of exactly the number of bytes that were received and then use AddRange() and get the result I want, but it just seems stupid to me. Not to mention it would be creating then throwing away an array on each read of data, which wouldn't be good for performance on a scalable multiuser server.

Is there a way to do this with a List? Or is there some other data structure I can use?

Answer 1

I don't know what protocol you are using, or if you are implementing a custom protocol, but if you identify the size you can use Buffer.BlockCopy to directly copy the bytes to a new array to add to your list.

It's hard to be more concise when you don't have specifics.

Answer 2

For .Net3.5 you can use the .Take() extension method to only return the actual number of bytes you received.

Answer 3

You could implement your own IEnumerable implementation which retrieves only the bytes you want from the array. Then you could do:

List.AddRange(new BufferEnumerator(Buffer));

Edit

You can also look at:

new System.ArraySegment(Buffer,0,numBytesRecieved)

I'm not positive if ArraySegment would work I remember reading some downsides of it but don't remember the specifics.

Answer 4

If you're using C# 3.5 (LINQ)

list.AddRange(buffer.Take(count));

Answer 5

You can use Array.Copy() and use only arrays to build your target buffer:

byte[] recvBuffer = new byte[1024];
byte[] message = new byte[0];
int nReaded;

while ((nReaded = ....Read(recvBuffer, 1024) > 0)
{
  byte[] tmp = new byte[message.Length + nReaded];
  Buffer.BlockCopy(message, 0, tmp, 0, message.Length);
  Buffer.BlockCopy(recvBuffer, 0, tmp, message.Length, nReaded);
  message = tmp;
}

EDIT: Replaced Array.Copy() with Buffer.BlockCopy() like suggested by Quintin Robinson in the comments.

Answer 6

Do you actually need the result to be a List<byte>? What are you going to do with it afterwards? If you really only need an IEnumerable<byte> I'd suggest creating something like this:

using System;
using System.Collections;
using System.Collections.Generic;

public class ArraySegmentConcatenator<T> : IEnumerable<T>
{
    private readonly List<ArraySegment<T>> segments =
        new List<ArraySegment<T>>();

    public IEnumerator<T> GetEnumerator()
    {
        foreach (ArraySegment<T> segment in segments)
        {
            for (int i=0; i < segment.Count; i++)
            {
                yield return segment.Array[i+segment.Offset];
            }
        }
    }

    public void Add(ArraySegment<T> segment)
    {
        segments.Add(segment);
    }

    public void Add(T[] array)
    {
        segments.Add(new ArraySegment<T>(array));
    }

    public void Add(T[] array, int count)
    {
        segments.Add(new ArraySegment<T>(array, 0, count));
    }

    public void Add(T[] array, int offset, int count)
    {
        segments.Add(new ArraySegment<T>(array, offset, count));
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }
}

Then you can just add the relevant segments each time. Of course, you could end up with a lot of wasted memory, and you'd have to be careful to create a new buffer each time (instead of reading over the original again) but it would be efficient in other ways.