Performance issues when reading a file from HD that dosen't follow computer byte alignments

Question

I have a file format (.STL, stereo lithography, the structure is standard and can not be changed. Not to be confused with standard template library) that uses a 50 byte data structure. Reading from the HD directly leads to wrong data being read as 50 bytes is not a multiple of 4.

Out of the entire 50 byte structure, I only need 36 of those bytes. The method I am using right now to extract the data is to manually offset the read position of the file to the point where the next set of data begins, read off the 36 bytes into a temporary variable and then dump the data in the temporary variable into it's proper position on the array.

Here's the code chunk:

threePoints* output = new threePoints [numTriangles]; // create an array to hold the entire file
threePoints* temp = new threePoints [1]; // temp variable to pull data out of each "cell"

// extract each triangle individualy
for (int i = 0; i < numTriangles; i++)
{
    stlFile.seekg (96 + i * 50, ios::beg);  //read vertex data and put them into tempoary array
                                            // offset = 80 header + 4 #triangles + 12 normal vector
    stlFile.read(reinterpret_cast<char*>(temp), (36)); // read the next 36 data blocks into holder, 3 points * 3 axis * 4 bytes per float
    output[i] = temp[0]; // dump values in holder into proper array
}

This method works but is slow. How would I go about doing this to make it faster and more efficiently?

Edit: I know manually disabling the byte alignment would solve this problem but I need to use the resulting array intensively and iterate over it many times. I've been told that disabling byte alignment can result in performance issues, thus i avoided it.

Answer 1

Instead of doing a bunch of small seeks & reads, allocate a large buffer (say, enough for 1000 structures) and read a bunch of records at once. (Then iterate through that buffer copying out the 36-byte chunks you need).

Answer 2

Bracket the struct definition with #pragma pack(1,push) and #pragma pack(pop). This is supported by most compilers. Then no automatic padding should be added.

The GCC manual claims it's a Microsoft compatibility feature, but it's been around forever… probably longer than GCC itself.

Answer 3

I don't have STL data to test with but how about something like this?...

#include <fstream>

#pragma pack(push, 1)

struct stl_hdr {
   unsigned int header[80];
   unsigned int triangle_cnt;
};

struct triangle {
   float norm;
   float v1;
   float v2;
   float v3;
   short attr_cnt;
};

#pragma pack(pop)

using namespace std;

int main(int argc, char* argv[]) {
   ifstream file(argv[1], ios::in | ios::binary);

   if (file.is_open()) {
      stl_hdr hdr;
      file.read((char*)&hdr, sizeof(hdr));

      triangle* tris = new triangle[hdr.triangle_cnt];

      file.read((char*)tris, hdr.triangle_cnt * sizeof(triangle));

      file.close();
   }

   return 0;
}

I created two structs based on Wikipedia's description of the STL format. The code above makes the same assumption that your code did...the attribute count is zero. And I left out error handling / cleanup.