Append 2+ byte arrays in C#

Question

Is there a best(see below) way to append two byte arrays in C#?

Pretending I have complete control, I can make the first byte array sufficiently large to hold the second byte array at the end and use the Array.CopyTo function. Or I can loop over individual bytes and make an assignment.

Are there better ways? I can't imagine doing something like converting the byte arrays to string and joining them and converting them back would be better than either method above.

In terms of best/better (in order):
1) Fastest
2) Least RAM consumption

Edit : A constraint is that I must work in the 2.0 framework .

Edit2 : The two choices recommended are MemoryStream and BlockCopy. I have run a simple speed test of 10,000,000 loops 3 times, and got the following results:

Average of 3 runs of 10,000,000 loops in milliseconds:

BlockCopy Time: 1154, with a range of 13 milliseconds
MemoryStream GetBuffer Time: 1470, with a range of 14 milliseconds
MemoryStream ToArray Time: 1895, with a range of 3 milliseconds
CopyTo Time: 2079, with a range of 19 milliseconds
Byte-by-byte Time: 2203, with a range of 10 milliseconds

Edit3 : Added results of List AddRange over 10 million loops:
List Time: 16694

Edit 4: Relative RAM Consumption (1 is baseline, higher is worse):

Byte-by-byte: 1
BlockCopy: 1
Copy To: 1
MemoryStream GetBuffer: 2.3
MemoryStream ToArray: 3.3
List: 4.2

The test shows that in general, unless you are doing a lot of byte copies [which I am], looking at byte copies is not worth a focus [e.g. 10 million runs yielding a difference of as much as 1.1 seconds].

Answer 1

Your first option of making the first array large enough to contain the second array and using Array.CopyTo ends up being roughly the same as manually iterating over each item and making the assignment. Array.CopyTo() just makes it more concise.

Converting to string and back to array will be horribly slow in contrast to the above. And would likely use more memory.

Answer 2

If you have arrays where the size will change from time to time, you're probably better off using a List<T> in the first place. Then you can just call the AddRange() method of the list.

Otherwise, Array.Copy() or Array.CopyTo() are as good as anything else you're likely to see.

Answer 3

You want BlockCopy

According to this blog post it is faster than Array.CopyTo.

Answer 4

Another option, although I haven't tested it to see how it fares in terms of speed and memory consumption, would the LINQ approach:

byte[] combined = bytesOne.Concat(bytesTwo).Concat(bytesThree).ToArray();

...where bytesOne, bytesTwo, and bytesThree are byte arrays. Since Concat uses deferred execution, this shouldn't create any intermediate arrays, and it shouldn't duplicate the original arrays until it constructs the final merged array at the end.

Edit: LINQBridge will allow you to use LINQ-to-Objects (which this is an example of) in the 2.0 framework. I understand if you don't want to depend on this, but it's an option.

Answer 5

Have you taught about using List or ArrayList instead of an Array? With these types they can grow or shrink and append via InsertRange

Answer 6

Create a new MemoryStream passing into the constructor a buffer that's exactly the size of the merged one. Write the individual arrays, and then finally use the buffer:

byte[] deadBeef = new byte[] { 0xDE, 0xAD, 0xBE, 0xEF};
byte[] baadF00d = new byte[] { 0xBA, 0xAD, 0xF0, 0x0D};
int newSize = deadBeef.Length + baadF00d.Length;
var ms = new MemoryStream(new byte[newSize], 0, newSize, true, true);
ms.Write(deadBeef, 0, deadBeef.Length);
ms.Write(baadF00d, 0, baadF00d.Length);
byte[] merged = ms.GetBuffer();

A lot of the low-level I/O functions in .NET take byte arrays and offsets. This was done to prevent needless copies. Be sure you really need the merged array if this is performance sensitive, otherwise just use buffers and offsets.

Answer 7

Do you need the output to actually be a byte array?

If not, you could create yourself a "smart cursor" (which is similar to what LINQ does): Create a custom IEnumerator<byte> that will first iterate the first array, and just continue on the second one without interuption.

This would work in the 2.0 framework be fast (in that the joining of arrays has virtually no cost), and use no more RAM than the arrays already consume.

Answer 8

You could also use an approach with a MemoryStream. Suppose b1 and b2 are two byte arrays, you can get a new one, b3, by using the MemoryStream in the following fashion:

var s = new MemoryStream();
s.Write(b1, 0, b1.Length);
s.Write(b2, 0, b2.Length);
var b3 = s.ToArray();

This should work without LINQ and is in fact quite a bit faster.