Reading data from hard drive into a class

Question

Hello everyone,

Every time i try to read a file form the hard drive and cast the data into a structure, i end up with problems of the data not casting properly. Is there a requirement with the reinterpret_cast() function that requires the number of bytes in a structure be a multiple of 4 bytes? If not, what am I doing wrong? If so, how do i get around that?

my structure looks like this: (they are in 50 byte chunks)

class stlFormat
{
public:

    float normalX, normalY, normalZ;
    float x1,y1,z1;
    float x2,y2,z2;
    float x3,y3,z3;

    char byte1, byte2;
};

Rest of my code:

void main()
{

int size;
int numTriangles;

int * header = new int [21]; // size of header

ifstream stlFile ("tetrahedron binary.STL", ios::in|ios::binary|ios::ate);

size = stlFile.tellg(); // get the size of file

stlFile.seekg(0, ios::beg); //read the number of triangles in the file
stlFile.read(reinterpret_cast<char*>(header), 84);

numTriangles = header[20];

stlFormat * triangles = new stlFormat [numTriangles]; //create data array to hold vertex data

stlFile.seekg (84, ios::beg); //read vertex data and put them into data array
stlFile.read(reinterpret_cast<char*>(triangles), (numTriangles * 50));

cout << "number of triangles: " << numTriangles << endl << endl;

for (int i = 0; i < numTriangles; i++)
{
    cout << "triangle " << i + 1 << endl;
    cout << triangles[i].normalX << " " << triangles[i].normalY << " " << triangles[i].normalZ << endl;
    cout << triangles[i].x1 << " " << triangles[i].y1 << " " << triangles[i].z1 << endl;
    cout << triangles[i].x2 << " " << triangles[i].y2 << " " << triangles[i].z2 << endl;
    cout << triangles[i].x3 << " " << triangles[i].z3 << " " << triangles[i].z3 << endl << endl;  
}

stlFile.close();
getchar();
}

Just for you John, although its rather incomprehensible. Its in hex format.

73 6f 6c 69 64 20 50 61 72 74 33 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 04 00 00 00 ec 05 51 bf ab aa aa 3e ef 5b f1 be 00 00 00 00 00 00 00 00 f3 f9 2f 42 33 33 cb 41 80 e9 25 42 9a a2 ea 41 33 33 cb 41 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ab aa aa 3e ef 5b 71 3f 33 33 4b 42 00 00 00 00 f3 f9 2f 42 33 33 cb 41 80 e9 25 42 9a a2 ea 41 00 00 00 00 00 00 00 00 f3 f9 2f 42 00 00 ec 05 51 3f ab aa aa 3e ef 5b f1 be 33 33 cb 41 00 00 00 00 00 00 00 00 33 33 cb 41 80 e9 25 42 9a a2 ea 41 33 33 4b 42 00 00 00 00 f3 f9 2f 42 00 00 00 00 00 00 00 00 80 bf 00 00 00 00 33 33 cb 41 00 00 00 00 00 00 00 00 33 33 4b 42 00 00 00 00 f3 f9 2f 42 00 00 00 00 00 00 00 00 f3 f9 2f 42 00 00

Answer 1

Most likely, float has an alignment of four bytes on your system. This means that, because you use it in your structure, the compiler will make sure the start of the structure when allocated using normal methods will always be a multiple of four bytes. Since the raw size of your structure is 4*12+2 = 50 bytes, it needs to be rounded up to the next multiple of four bytes - otherwise, the second element of arrays of this structure would be unaligned. So your struct ends up 52 bytes, throwing off your parsing.

If you need to parse a binary format, it's often a good idea to either use compiler-specific directives to disable alignment, or read one field at a time, to avoid these problems.

For example, on MSVC++, you can use __declspec(align(1)) Edit: Actually __declspec(align(X)) can only increase alignment restrictions. Oops. You'll need to either load one field at a time, or make the padding part of the binary format.

Answer 2

instead of fiddling with padding and differences between platforms, maybe have a look at serialization to/from binary files? It might be somewhat less performant then reading data straight into memory, but it's way more extensible.

Answer 3

I used my favorite text editor (editpadpro) to save the file you posted in the OP as a binary file called "c:\work\test.bin", edited your code to the following, and it (apparently) produced the correct (expected) output. Please try it out.

#include <cstdlib>
#include <iostream>
#include <fstream>
using namespace std;

#pragma pack( push, 1 )
class stlFormat
{
public:

    float normalX, normalY, normalZ;
    float x1,y1,z1;
    float x2,y2,z2;
    float x3,y3,z3;

    char byte1, byte2;
};
#pragma pack( pop ) 


struct foo
{
    char c, d, e;
};

void main()
{

    size_t sz = sizeof(foo);

int size;
int numTriangles;

int * header = new int [21]; // size of header

ifstream stlFile ("c:\\work\\test.bin", ios::in|ios::binary|ios::ate);

size = stlFile.tellg(); // get the size of file

stlFile.seekg(0, ios::beg); //read the number of triangles in the file
stlFile.read(reinterpret_cast<char*>(header), 84);

numTriangles = header[20];

stlFormat * triangles = new stlFormat [numTriangles]; //create data array to hold vertex data

stlFile.seekg (84, ios::beg); //read vertex data and put them into data array
stlFile.read(reinterpret_cast<char*>(triangles), (numTriangles * 50));

cout << "number of triangles: " << numTriangles << endl << endl;

for (int i = 0; i < numTriangles; i++)
{
    cout << "triangle " << i + 1 << endl;
    cout << triangles[i].normalX << "   " << triangles[i].normalY << "  " << triangles[i].normalZ << endl;
    cout << triangles[i].x1 << "    " << triangles[i].y1 << "   " << triangles[i].z1 << endl;
    cout << triangles[i].x2 << "    " << triangles[i].y2 << "   " << triangles[i].z2 << endl;
    cout << triangles[i].x3 << "    " << triangles[i].z3 << "   " << triangles[i].z3 << endl << endl;       
}

stlFile.close();
getchar();
}

Answer 4

You should be aware that you are throwing portability out the window with that kind of code: your files may be incompatible with new versions of your program if you compile with a different compiler or for a different system.

That said, you might fix this by using sizeof( int[21] ) and sizeof( stlFormat[ numTriangles ] ) rather than hardcoded sizes in bytes. Reason being, as others noted, the alignment bytes your compiler may or may not add.

If this is a program that other people may use or files might be shared, look up serialization.

Answer 5

I think the problem is not so much the reading of each individual triangle as that the triangle array isn't laid out as you think. There appear to be 50 bytes in each struct, but the allocated memory is almost certainly laid out as if the structs were 52 bytes. Consider reading in each struct individually.

Two more points:

First, there is no such thing as void main in C++. Use int main().

Second, you seem to be leaking memory. You'd be better off in general using the vector facility.

Answer 6

IMO you really ought to be explicitly reading the triangles directly (deserialization) instead of casting bytes. Doing so will help you avoid portability and performance problems. If you're doing a lot of calculations with those triangles after you read them, the performance hit for using a non-standard memory layout can be non-trivial.

Replace the line "stlFile.read(reinterpret_cast(triangles), (numTriangles * 50));" with this:

for (int i = 0; i < numTriangles; i++)
{
  stlFile.read((char*)&triangles[i].normalX, sizeof(float));
  stlFile.read((char*)&triangles[i].normalY, sizeof(float));
  stlFile.read((char*)&triangles[i].normalZ, sizeof(float));
  stlFile.read((char*)&triangles[i].x1, sizeof(float));
  stlFile.read((char*)&triangles[i].y1, sizeof(float));
  stlFile.read((char*)&triangles[i].z1, sizeof(float));
  stlFile.read((char*)&triangles[i].x2, sizeof(float));
  stlFile.read((char*)&triangles[i].y2, sizeof(float));
  stlFile.read((char*)&triangles[i].z2, sizeof(float));
  stlFile.read((char*)&triangles[i].x3, sizeof(float));
  stlFile.read((char*)&triangles[i].y3, sizeof(float));
  stlFile.read((char*)&triangles[i].z3, sizeof(float));
  stlFile.read(&triangles[i].byte1, 1);
  stlFile.read(&triangles[i].byte2, 1);
}

It takes a little more code and a little more time to read in the triangles, but you'll avoid a few potential headaches.

Note that writing triangles also requires similar code to avoid inadvertently writing out some padding.

Answer 7

Storing a struct entirely at once isn't portable unless you take great care with compiler-specific flags and all compilers and architectures might still not allow the same binary format. Storing a field (e.g. a floating-point number) at a time is better, but still isn't portable because of endianess issues and possibly different data types (e.g. what is sizeof(long) on your system).

In order to save integers safely and portably, you have to format them byte at a time into a char buffer that will then be written out to a file. E.g.

char buf[100];  // Extra space for more values (instead of only 4 bytes)
// Write a 32 bit integer value into buf, using big endian order
buf[0] = value >> 24;  // The most significant byte
buf[1] = value >> 16;
buf[2] = value >> 8;
buf[3] = value;  // The least significant byte

Similarly, reading back has to be done a byte at a time:

// Converting the pointer to unsigned to avoid sign extension issues
unsigned char* ubuf = reinterpret_cast<unsigned char*>(buf);
value = ubuf[0] << 24 | ubuf[1] << 16 | ubuf[2] << 8 | ubuf[3];

If little endian order is desired, invert the indexing order of buf and ubuf.

Because no pointer casting of integer types to char or vice-versa are done, the code is fully portable. Doing the same for floating-point types requires extra caution and a pointer cast so that the value can be handled as an integer, so that bit shifting works. I won't cover that in detail here.

While this solution seems extremely painful to use, you only need to write a few helper functions to make it tolerable. Alternatively, especially if the exact format used does not matter to you, you can use an existing serialization library. Boost.Serialization is a rather nice library for that.