Hi folks, When calculating a golf handicap differential you are supposed to truncate the answer to 1 decimal place without rounding. No idea why but... I know how to do this using TRUNCATE() in mySQL SELECT TRUNCATE( 2.365, 1 ); // outputs 2.3 but I was wondering if sprintf() could do this? The only way I know to work with decim...
Hi all, I want to use number_format function in PHP. For example: $number = 234.51; echo number_format($number,2); This works for float numbers but I want to use it for different numbers. If a number is decimal and doesn't have any floating points, it shows like : 145.00 . How can I fix this? I mean I want to show as many floating po...
Hi everyone, I did some speed testing to figure out what is the fastest, when doing multiplication or division on numbers. I had to really work hard to defeat the optimiser. I got nonsensical results such as a massive loop operating in 2 microseconds, or that multiplication was the same speed as division (if only that were true). After...
>>> float(str(0.65000000000000002)) 0.65000000000000002 >>> float(str(0.47000000000000003)) 0.46999999999999997 ??? What is going on here and how do I convert 0.47000000000000003 to string and the resultant value back to float? I am using python 2.5.4 on windows. ...
Hi all, I have a function like below: void add(int&,float&,float&); and when I call: add(1,30,30) it does not compile. add(1,30.0,30.0) also does not compile. It seems that in both cases, it gets implicitly converted to double instead of float. So, do you suggest that it is better to re-define add as add(int&,double&,double&)? Is...
Is there any way in Haskell to get the constant that is the largest and smallest possible positive rational number greater than zero that can be represented by doubles? ...
How to convert a floating point number into a sequence of bytes so that it can be persisted in a file? Such algorithm must be fast and highly portable. It must allow also the opposite operation, deserialization. It would be nice if only very tiny excess of bits per value (persistent space) is required. ...
I want to use lua (that internally uses only doubles) to represent a integer that can't have rounding errors between 0 and 2^64-1 or terrible things will happen. Is it possible to do so? ...
I'm having problems parsing a hex formatted DEC 32bit single precision floating point value in python, the value I'm parsing is represented as D44393DB in hex. The original floating point value is ~108, read from a display of the sending unit. The format is specified as: 1bit sign + 8bit exponent + 23bit mantissa. Byte 2 contains the si...
I'm creating a financial app and it seems my floats in sqlite are floating around. Sometimes a 4.0 will be a 4.000009, and a 6.0 will be a 6.00006, things like that. How can I make these more exact and not affect my financial calculations? Values are coming from Python if that matters. Not sure which area the messed up numbers are co...
I am converting four bytes to float and I'm getting NaN as a result, but I want the value 0.0. What am I doing wrong? This is my code: public class abc { public static void main(String[] args) { int[] arry = { 255, 255, 255, 255 }; int num = ((arry[0] << 24) & 0xFF000000) | ((arry[1] << 16) & 0xFF0000) | ((...
Hello all, It is often hard to find the origin of a NaN, since it can happen at any step of a computation and propagate itself. So is it possible to make a C++ program halt when a computation returns NaN or inf? The best in my opinion would be to have a crash with a nice error message: Foo: NaN encoutered at Foo.c:624 Is something li...
So, I'm having some precision issues in Python. I would like to calculate functions like this: P(x,y) = exp(-x)/(exp(-x) + exp(-y)) Where x and y might be >1000. Python's math.exp(-1000) (in 2.6 at least!) doesn't have enough floating point precision to handle this. this form looks like logistic / logit / log-odds, but it's not...
Of course one should never compare floating point values that result from a calculation for equality, but always use a small tolerance, e.g.: double value1 = ... double value2 = ... if (Math.Abs(value1 - value2) < tolerance * Math.Abs(value1)) { ... values are close enough } But if I use Math.Round can I always be sure that the r...
Is there a way to set Perl script's floating point precision (to 3 digits), without having to change it specifically for every variable? Something similar to TCL's: global tcl_precision set tcl_precision 3 ...
char byte_to_ascii(char value_to_convert, volatile char *converted_value) { if (value_to_convert < 10) { return (value_to_convert + 48); } else { char a = value_to_convert / 10; double x = fmod((double)value_to_convert, 10.0); char b = (char)x; a = a + 48; b = b + 48; *converted_value = a; *(converted_value+1) = b; r...
I have this XML data and try and make a sum of it using the XSLT snippet below. Xml <?xml version="1.0" encoding="utf-8"?> <values> <value>159.14</value> <value>-2572.50</value> <value>-2572.50</value> <value>2572.50</value> <value>2572.50</value> <value>-159.14</value> </values> Xslt <?xml version="1.0" en...
I need to format float value and I only need 2 numbers after point and should be rounded value float first = 7, Second = 3,result; result = first / Second; // result contain 2.33333325 since I need like 2.33 Thanks ...
I'm working on a Java/Groovy program. I have a double variable that holds a number that was typed in by a user. What I really want to know is how many numbers the user typed to the right of the decimal place. Something like: double num = 3.14 num.getPlaces() == 2 Of course, you can't do this with a double since that's using IEEE flo...
i am developing a code where i have to load floating point values stored in each line at a time in a text file...i loaded each of these data into a array of floats using fscanf()...however i found that the floating points were stored in a different way, example 407.18 was stored as 407.179993, 414.35 as 414.350006...now i am stuck becaus...