Possible Duplicate: Prevent scientific notation in ostream when using << with double I get 1e-1 as result after a computation how can I convert the result from exponent to dot notation i.e., 0.1 ? Why is it automatically converted to exponential notation!! ...
Long: char long_num[8]; for(i=0; i<8; i++) long_num[i] = data[pos++]; memcpy(&res, long_num, 8); The values in the long_num are as follows: 127 -1 -1 -1 -1 -1 -1 -1 res should be the maximum value of signed long, but is -129 instead. EDIT: This one is taken care of. It was a result of communication problems: For the pers...
Floating point values are inexact, which is why we should rarely use strict numerical equality in comparisons. For example, in Java this prints false (as seen on ideone.com): System.out.println(.1 + .2 == .3); // false Usually the correct way to compare results of floating point calculations is to see if the absolute difference agains...
I know it's an odd question, but does JavaScript have the capacity to work with double's as opposed to single floats? (64 bit floats vs. 32 bits.) ...
i successfully complied the code: #include <stdio.h> #include <math.h> int q; int main() { srand( time(NULL) ); int n=3; q=ceil(sqrt(n)); printf("%d\n %d\n", n,q); if(n == 2) printf("%d\n is prime", n); else if(n % 2 == 0.0 || n < 2) printf("%d\n is not prime", n); else { int x;...
I am going to do some math calculations using C++ . The input floating point number is a valid number, but after calculations , the resulting value becomes NaN. I would like to trace the point where NaN value appears (possibly using gdb), instead of inserting a lot of isNan() into the code. But, I found that even code like this wil...
What is a platform-independent way of specifying the largest representable negative floating-point number? We found an algorithm that broke when run on a PS3's SPU, but worked fine when compiled for the PPU: float x = -FLT_MAX; /* stuff */ if (x > 0.0f) { // If x is unchanged, code is executed on SPU } Essentially, is there a wel...
I recently ran into an issue where I wasn't getting the numerical result I expected. I tracked it down to the problem that is illustrated by the following example: #include <stdio.h> int main() { double sample = .5; int a = (int)(sample * (1 << 31)); int b = (int)(sample * (1 << 23) * (1 << 8)); printf("a = %#08x, b = %#08x\n",...
What's the best way to test two Singles for equality in VB6? I want to test two Single values for equality to 7 significant figures. This MSDN article recommends using something like If Abs(a - b) <= Abs(a / 10 ^ 7) Then valuesEqual = True End If However, that can fail for certain values, e.g. Public Sub Main() Dim a As S...
Alternative wording: When will adding Double.MIN_VALUE to a double in Java not result in a different Double value? (See Jon Skeet's comment below) This SO question about the minimum Double value in Java has some answers which seem to me to be equivalent. Jon Skeet's answer no doubt works but his explanation hasn't convinced me how it...
Possible Duplicate: How is floating point stored? When does it matter? I was wondering what's the reason for the next behavior in Python: >>> print(0.1 + 0.1 + 0.1 - 0.3) 5.55111512313e-17 To fix that one must use decimal instead: >>> from decimal import Decimal >>> Decimal('0.1') + Decimal('0.1') + Decimal('0.1') - Decim...
I recieved a crash report from MadExcept from a user. The Exception was Invalid floating point operation. The odd part though is that the callstack dies at @FSafeDivide. I did a google and found out that this was a check for certain pentium chips which didn't do division correctly. If the test failed all the divisions would be done i...
I'm somewhat familiar with the x87 instructions for manipulating floating point numbers in x86 assembly. However, I read somewhere that these were seldom used anymore. (And weren't allowed in 64-bit Windows drivers)[1] If that's the case, what instructions should I be using? I saw something about SSE, but unless I'm mistaken, those inst...
If I compare two floating-point numbers, are there cases where a>=b is not equivalent to b<=a and !(a<b), or where a==b is not equivalent to b==a and !(a!=b)? ...
If I have a function f that computes element m of a sequence of digits in base b, is it in general possible to write a function g that computes element n of the corresponding sequence in base c ? As a contrived example, say f produces binary and g produces hexadecimal: f(m) 1, 0, 1, 0, 1, 0, 1, 0, ... g(n) A, A, ... Now say f is i...
In C++, for example fmod(-2,2) returns -0. The expression -0 == 0 is true, but the bits are different. What is the purpose of having something like -0 which should be 0 but is represented differently? Is -0 used exactly the same way as 0 in any computations? ...
I just recently came across the Kahan (or compensated) summation algorithm for minimizing roundoff, and I'd like to know if there are equivalent algorithms for division and/or multiplication, as well as subtraction (if there happens to be one, I know about associativity). Implementation examples in any language, pseudo-code or links woul...
$a = '35'; $b = '-34.99'; echo ($a + $b); Results in 0.009999999999998 What is up with that? I wondered why my program kept reporting odd results. Why doesn't PHP return the expected 0.01? ...
I get this code snippet from some where else. According to the webmaster, the code is picked from The art of computer programming by Knuth Since I do not have a copy of that book, may I know what is the difference among the two functions? bool approximatelyEqual(float a, float b, float epsilon) { return fabs(a - b) <= ( (fabs(a) < ...
So I've always been told NEVER to do this, and this time I pose the question to you: why? I'm sure there is a very good reason, I simply do not know what it is. :-P ...