Reading in a binary file containing an unknown quantity of structures (C#)

Question

Ok, so I currently have a binary file containing an unknown number of structs like this:

private struct sTestStruct
{
    public int numberOne;
    public int numberTwo;
    public int[] numbers; // This is ALWAYS 128 ints long.
    public bool trueFalse;
}

So far, I use the following to read all the structs into a List<>:

List<sTestStruct> structList = new List<sTestStruct>();

while (binReader.BaseStream.Position < binReader.BaseStream.Length)
{
    sTestStruct temp = new sTestStruct();
    temp.numberOne = binReader.ReadInt32();
    temp.numberTwo = binReader.ReadInt32();
    temp.numbers = new int[128];
    for (int i = 0; i < temp.numbers.Length; i++)
    {
        temp.numbers[i] = binReader.ReadInt32();
    }
    temp.trueFalse = binReader.ReadBoolean();

    // Add to List<>
    structList.Add(temp);
}

I don't really want to do this, as only one of the structs can be displayed to the user at once, so there is no point reading in more than one record at a time. So I thought that I could read in a specific record using something like:

fileStream.Seek(sizeof(sTestStruct) * index, SeekOrigin.Begin);

But it wont let me as it doesn't know the size of the sTestStruct, the structure wont let me predefine the array size, so how do I go about this??

Answer 1

The sTestStruct is not stored in one consecutive are of memory and sizeof(sTestStruct) is not directly related to the size of the records in the file. The numbers members is a reference to an array which you allocate youself in your reading code.

But you can easily specify the record size in code since it is a constant value. This code will seek to the record at index. You can then read one record using the body of your loop.

const Int32 RecordSize = (2 + 128)*sizeof(Int32) + sizeof(Boolean);
fileStream.Seek(RecordSize * index, SeekOrigin.Begin); 

If you have many different fixed sized records and you are afraid that manually entering the record size for each record is error prone you could devise a scheme based on reflection and custom attributes.

Create an attribute to define the size of arrays:

[AttributeUsage(AttributeTargets.Field, AllowMultiple = false)]
sealed class ArraySizeAttribute : Attribute {

  public ArraySizeAttribute(Int32 length) {
    Length = length;
  }

  public Int32 Length { get; private set; }

}

Use the attribute on your record type:

private struct sTestStruct {   
  public int numberOne;   
  public int numberTwo;   
  [ArraySize(128)]
  public int[] numbers; // This is ALWAYS 128 ints long.   
  public bool trueFalse;   
}

You can then compute the size of the record using this sample code:

Int32 GetRecordSize(Type recordType) {
  return recordType.GetFields().Select(fieldInfo => GetFieldSize(fieldInfo)).Sum();
}

Int32 GetFieldSize(FieldInfo fieldInfo) {
  if (fieldInfo.FieldType.IsArray) {
    // The size of an array is the size of the array elements multiplied by the
    // length of the array.
    var arraySizeAttribute = (ArraySizeAttribute) Attribute.GetCustomAttribute(fieldInfo, typeof(ArraySizeAttribute));
    if (arraySizeAttribute == null)
      throw new InvalidOperationException("Missing ArraySizeAttribute on array.");
    return GetTypeSize(fieldInfo.FieldType.GetElementType())*arraySizeAttribute.Length;
  }
  else
    return GetTypeSize(fieldInfo.FieldType);
}

Int32 GetTypeSize(Type type) {
  if (type == typeof(Int32))
    return 4;
  else if (type == typeof(Boolean))
    return 1;
  else
    throw new InvalidOperationException("Unexpected type.");
}

Use it like this:

var recordSize = GetRecordSize(typeof(sTestStruct));
fileStream.Seek(recordSize * index, SeekOrigin.Begin); 

You will probably have to expand a little on this code to use it in production.

Answer 2

From everything I have read, the way you are doing it is the best method to read in binary data as it has the fewest gotchas where things can go wrong.

Answer 3

Define your struct like this:

struct sTestStruct
{
    public int numberOne;
    public int numberTwo;
    [MarshalAs(UnmanagedType.ByValArray, SizeConst=128)]
    public int[] numbers; // This is ALWAYS 128 ints long. 
    public bool trueFalse;
}

And use Marshal.Sizeof(typeof(sTestStruct)).