Decimal precision of floats

Question

equivalent to log10(2^24) ≈ 7.225 decimal digits

Wikipedia

Precision: 7 digits

MSDN

6

std::numeric_limits<float>::digits10

Why numeric_limits return 6 here? Both Wikipedia and MSDN report that floats have 7 decimal digits of precision.

Answer 1

It's really only 23 bits in the mantissa (there's an implied 1, so it's effectively 24 bits, but the 1 obviously does not vary). This gives 6.923689900271567 decimal digits of precision, which is not quite 7.

Answer 2

If in doubt, read the spec. The C++ standard says that digits10 is:

Number of base 10 digits that can be represented without change.

That's a little vague; fortunately, there's a footnote:

Equivalent to FLT_DIG, DBL_DIG, LDBL_DIG

Those are defined in the C standard; let's look it up there:

number of decimal digits, q, such that any floating-point number with q decimal digits can be rounded into a floating-point number with p radix b digits and back again without change to the q decimal digits.

So std::numeric_limits<float>::digits10 is the number of decimal digits such that any floating-point number with that many digits is unchanged if you convert it to a float and back to decimal.

As you say, floats have about 7 digits of decimal precision, but the error in representation of both fixed-width decimals and floats is not uniformly logarithmic. The relative error in rounding a number of the form 1.xxx.. to a fixed number of decimal places is nearly ten times larger than the relative error of rounding 9.xxx.. to the same number of decimal places. Similarly, depending on where a value falls in a binade, the relative error in rounding it to 24 binary digits can vary by a factor of nearly two.

The upshot of this is that not all seven-digit decimals survive the round trip to float and back, but all six digit decimals do. Hence, std::numeric_limits<float>::digits10 is 6.

There are not that many six and seven digit decimals with exponents in a valid range for the float type; you can pretty easily write a program to exhaustively test all of them if you're still not convinced.