## floating-point

### A little diversion into floating point (im)precision, part 1

Most mathematicians agree that e ** (πi) + 1 = 0. However, most floating point implementations disagree. How well can we settle this dispute? I'm keen to hear about different languages and implementations, and various methods to make the result as close to zero as possible. Be creative! ...

### Convert Bytes to Floating Point Numbers in Python

I have a binary file that I have to parse and I'm using Python. Is there a way to take 4 bytes and convert it to a single precision floating point number?...

### Most effective way for float and double comparison

What would be the most efficient way to compare two doubles or two floats (single precision)? Simply doing this is not correct: bool CompareDoubles1 (double A, double B) { return A == B; } But something like: bool CompareDoubles2 (double A, double B) { diff = A - B; return (diff < EPSILON) && (-diff > EPSILON); } Seems t...

### Comparing IEEE floats and doubles for equality

What is the best method for comparing IEEE floats and doubles for equality? I have heard of several methods, but I wanted to see what the community thought. ...

### How do you generate a random number in C#?

I would like to generate a random floating point number between 2 values. What is the best way to do this in C#? ...

### round() in Python doesn't seem to be rounding properly

The documentation for the round() function states that you pass it a number, and the positions past the decimal to round. Thus is should do this: n = 5.59 round(n, 1) # 5.6 But, in actuality, good old floating point weirdness creeps in and you get: 5.5999999999999996 For the purposes of UI, I need to display '5.6'. I poked around t...

### How is floating point stored? When does it matter?

In follow up to this question, it appears that some numbers cannot be represented by floating point at all, and instead are approximated. How are floating point numbers stored? Is there a common standard for the different sizes? What kind of gotchas do I need to watch out for if I use floating point? Are they cross-language compatibl...

### Convert from scientific notation string to float in C#

What's the proper way to convert from a scientific notation string such as "1.234567E-06" to a floating point variable using C#? ...

### how to use "%f" to populate a double value into a string with the right precision

I am trying to populate a string with the double value using a sprintf like this sprintf(S, "%f", val); But the precision is being cut of to 6 decimal places. I need about 10 decimal places for the precision. Kindly tell me how that can be achieved. ...

### cout prints "-0" instead of "0"

#include <iostream> using namespace std; int main() { double u = 0; double w = -u; cout << w << endl; return 0; } Why does this great piece of code outputs "-0" and not "0", as one would expect? ...

### Determine if a string is an integer or a float in ANSI C

Using only ANSI C, what is the best way to, with fair certainty, determine if a C style string is either a integer or a real number (i.e float/double)? ...

### What is the fastest way to convert float to int on x86

What is the fastest way you know to convert a floating-point number to an int on an x86 CPU. Preferrably in C or assembly (that can be in-lined in C) for any combination of the following: 32/64/80-bit float -> 32/64-bit integer I'm looking for some technique that is faster than to just let the compiler do it. ...

### How to manually parse a floating point number from a string

Of course most languages have library functions for this, but suppose I want to do it myself. Suppose that the float is given like in a C or Java program (except for the 'f' or 'd' suffix), for example "4.2e1", ".42e2" or simply "42". In general, we have the "integer part" before the decimal point, the "fractional part" after the decima...

### Converting floating point exceptions into C++ exceptions

Is it possible to convert floating point exceptions (signals) into C++ exceptions on x86 Linux? This is for debugging purposes, so nonportability and imperfection is okay (e.g., if it isn't 100% guaranteed that all destructors are called). ...

### Double and floats in C#

Are you aware what precision of operations on floating point numbers is different in project compiled in debug mode vs the one compiled in release mode. ...

### Why would I use 2's complement to compare two doubles instead of comparing their differences against an epsilon value?

Referenced here and here...Why would I use two's complement over an epsilon method? It seems like the epsilon method would be good enough for most cases. Update: I'm purely looking for a theoretical reason why you'd use one over the other. I've always used the epsilon method. Has anyone used the 2's complement comparison successfu...

### SQL server 2005 numeric precision loss

Hello, Debugging some finance-related SQL code found a strange issue with numeric(24,8) mathematics precision. Running the following query on your MSSQL you would get A + B * C expression result to be 0.123457 SELECT A, B, C, A + B * C FROM ( SELECT CAST(0.12345678 AS NUMERIC(24,8)) AS A, CAST(0 AS NUMERIC(...

### Test if a floating point number is an integer

This code works (C# 3) double d; if(d == (double)(int)d) ...; Is there a better way to do this? For extraneous reasons I want to avoid the double cast so; what nice ways exist other than this? (even if they aren't as good) Note: Several people pointed out the (important) point that == is often problematic regrading floating point. ...

### Significant figures in the decimal module

So I've decided to try to solve my physics homework by writing some python scripts to solve problems for me. One problem that I'm running into is that significant figures don't always seem to come out properly. For example this handles significant figures properly: from decimal import Decimal >>> Decimal('1.0') + Decimal('2.0') Decima...

### How to alter a float by its smallest increment (or close to it)?

I have a double value f and would like a way to nudge it very slightly larger (or smaller) to get a new value that will be as close as possible to the original but still strictly greater than (or less than) the original. It doesn't have to be close down to the last bit—it's more important that whatever change I make is guaranteed to pro...