problem with casting float -> double in C when fread

Question

I have a problem with casting from float to double when fread;

fread(doublePointer,sizeofFloat,500,f);

if i change double pointer to float pointer, it works just fine. However,i need it to be double pointer for laster on, and i thought when i write from small data type (float)to bigger data type(double)'s memory, it should be fine. but it turns out it doesnt work as i expected. what is wrong with it, and how do i solve this problem.

i know i can solve it by converting it one by one. but i have a huge amount of data. and i dont wanna extra 9000000+ round of converting.. that would be very expensive. and is there any trick i can solve it?

is there any c++/c tricks

thanks

Answer 1

If you write float-formatted data into a double, you're only going to get garbage as a result. Sure, you won't overflow your buffer, but that's not the only problem - it's still going to be finding two floats where it expects a double. You need to read it as a float, then convert - casting (even implicitly) in this manner lets the compiler know that the data was originally a float and needs to be converted:

float temp[500];
int i;

fread(temp, sizeof(temp[0]), 500, f);
for (i = 0; i < 500; i++)
    doublePointer[i] = temp[i];

Answer 2

Suppose for example a float is 4 bytes on your computer. If you read 500 floats then you read 2000 bytes, one float per float and the result is correct.

Suppose for example a double is 8 bytes on your computer. If you read 500 floats then you read 2000 bytes, but you're reading them into 250 doubles, 2 floats per double, and the result is nonsense.

If your file has 500 floats you have to read 500 floats. Cast each float value to a double value. You can convert each numeric value that way.

When you abuse a pointer, pretending that the pointer points to a type of data that it doesn't really point to, then you're not converting each numeric value, you're preserving nonsense as nonsense.

Edit: You added to your question "and i dont wanna extra 9000000+ round of converting.. that would be very expensive. and is there any trick i can solve it?" The answer is yes, you can use a trick of keeping your floats as floats. If you don't want to convert to doubles then don't convert to doubles, just keep your floats as floats.

Answer 3

9000000 conversions from float to double is nothing. fread into a float array, then convert that into a double array.

Benchmark this code scientifically, don't guess about where the slowdowns might be.

If you're bottlenecked on the conversion, write a unrolled, vectorized conversion loop, or use one from a commercial vector library.

If it's still too slow, tile your reads so you read in your float data in batches of a few pages that fit in L1 cache, then convert those to double, then read the next few pages and convert those to double, etc.

If it's still too slow, investigate loading your data lazily so only the parts that are needed get loaded, and only when they are used.

A modern x86 core is capable of doing two float->double conversions per cycle in a hand-tuned vectorized loop; at 2GHz, that's 4 billion conversions per second per core. 9 million conversions is small change -- my laptop does it in less than 1 millisecond.

---

Alternatively, just convert the whole dataset to double once, and read it in that way from now on. Problem solved.

Answer 4

I would look at this from a different perspective. If the data is stored as float, then that is all the precision it will ever have. There is no point in converting to double until the rules of floating point arithmetic require it.

So I would allocate a buffer for the 500 (or whatever) floats, and read them from the data file with one suitable call to fread():

float *databuffer;
//...
databuffer = malloc(500 * sizeof(float));
fread(databuffer, sizeof(Float), 500, f);

Later, use the data in whatever math it needs to participate in. It will be promoted to double if required. Don't forget to eventually free the buffer after it is no longer needed.

If your results really do have all the precision of a double, then use a fresh buffer of doubles to hold them. However, if they are to be written back to file as float, then you will eventually need to put them into a buffer of floats.

Note that reading and writing files for interchange often needs to be considered a separate problem from efficient storage and usage of data in memory. It is often necessary to read a file and process each individual value in some way. For example, a portable program might be required to handle data written by a system using a different byte order. Less frequently today, you might find that even the layout of the bits in a float differs between systems. In general, this problem is often best solved by deferring to a library implementing a standard such as XDR (defined by RFC 4506) that was designed to deal with binary portability.