As (hopefully) most of you know, floating point arithmetic is different from real number arithmetic. It's for starters imprecise. Many numbers, especially decimals (0.1, 0.3) cannot be represented, leading to problems like this. A more thorough list can be found here. Are there any general purpose languages that have built-in support fo...
We have a situation we want to do a sort of weighted average of two values w1 & w2, based on how far two other values v1 & v2 are away from zero... for example: If v1 is zero, it doesn't get weighted at all so we return w2 If v2 is zero, it doesn't get weighted at all so we return w1 If both values are equally far from zero, we do a me...
I need to encode and decode IEEE 754 floats and doubles from binary in node.js to parse a network protocol. Are there any existing libraries that do this, or do I have to read the spec and implement it myself? Or should I write a C module to do it? ...
While learning about precision in floating point arithmetic and different methods to avoid it (using a conjugate, taylor series,...) books frequently mention the subtraction of two very similar numbers or one large and one small number as the biggest cause of error. How come it is only subtraction that causes this and not addition? As I ...
Inspired by this question, the following doesn't do what I'd expect it to: float myFloat = 0.6; Console.WriteLine(myFloat); // Output: 0.6 I'd expect the above to print out 0.60000002384185791 (the floating point representation of 0.6) - clearly there is some mechanism here which is making this work when in fact it shouldn't (although...
I have a bit of javascript that dynamically multiplies what users are typing in a text field (by the base var), and displays it in a Now I'm just trying to figure out how to get the decimal places of the result to float to 2 places, i.e. 10.00 instead of 10 I found the toFixed function, but can't seem to use it properly... I'd appreciat...
Let's say I want to write a function that does the following: Given a number N, when N is rounded to D1 digits does the result include more than D2 decimal places, not counting trailing zeroes? For example, say N is .01001, D1 is 4, and D2 is 2. The question becomes, does .0100 include more than 2 decimal places, not counting t...
What is the highest number this javascript expression can evaluate to? What is the lowest number? Why? +(''+Math.random()).substring(2) Extra credit: How many different values can the expression evaluate to? Can it be every value from the minimum to the maximum, or are some intermediate values not obtainable due to rounding issues? ...
I have g++ 4.4.3 on Linux with Ubuntu Lucid Lynx, and I am getting a: -nan as a result. On Hardy Heron with g++ 4.3.1, I am getting all nan This is causing my text diff regression to fail, since I am using cout to print this numerical result. What is the meaning of a signed nan, and is there a way to tell the compiler that an unsig...
I am writing a simple app that takes a bunch of numerical inputs and calculates a set of results. (The app is in PyGTK but I don't think that's relevant.) My problem is that if I want to just have NaN's and Inf's propagated through, then in every calculation I need to do something like: # At the top of the module nan = float("nan") inf...
Below is an excerpt from a program - float val=214.20; double val1=214.20; printf("\n\n\nfloat :: %f, %4.6f, %4.2f \n ",val,val,val); printf("double:: %f, %4.6f, %4.2f \n ",val1,val1,val1); And the output is - float :: 214.199997, 214.199997, 214.20<--- is the correct value as we wanted double:: 214.200000, 214.200000, 214.20...
Why does this: int main(void) { short w = 30; return 1.2 * w; } return 35? ...
If I wanted to convert a number Ex. 32.24x10^5 to IEEE 754 standard BY HAND how would I do it? ...
I've got some image processing code in C++ which calculates gradients and finds straight lines in them with the hough transformation algorithm. The program does most of the calculations with floats. When I run this code on the same image on two different computers, one Pentium IV running latest Fedora, the other a Core i5 latest Ubuntu...
I am new to python and I was writing something like: t = 0. while t<4.9: t = t + 0.1 if t == 1.: ... do something ... I noticed that the if statement was never being executed. So I modified the code to look like this: ''' Case a''' t = 0. while t<4.9: t = t + 0.1 print(t) print(t == 5.) When I run t...
int x = 2; volatile int y = 2; const int z = x/y; int main(){ int x = 2 + 3; double d = 7 / 3; } I have three questions here: Firstly, can the compiler calculate the value of the 'z' at compile time to be 1 in this case? Secondly, I observed that the compiler does not generate assembly instructions for adding 2 and 3 to in...
Trying out a problem of finding the first k digits of a num^num I wrote the same program in C++ and Python C++ long double intpart,num,f_digit,k; cin>>num>>k; f_digit= pow(10.0,modf(num*log10(num),&intpart)+k-1); cout<<f_digit; Python (a,b) = modf(num*log10(num)) f_digits = pow(10,b+k-1) print f_digits Input 19423474 9 Output...
Hello, I am new to programming, I've done web development, but I am currently trying to learn real programming. The question I have is already answered here. union ufloat { float f; unsigned u }; ufloat u1; u1.f = 0.3f; What I don't get is how it works. What does the 0.3 part do? I couldn't find it in my text. And how does this ...
As a part of some unit testing code that I'm writing, I wrote the following function. The purpose of which is to determine if 'a' could be rounded to 'b', regardless of how accurate 'a' or 'b' are. def couldRoundTo(a,b): """Can you round a to some number of digits, such that it equals b?""" roundEnd = len(str(b)) if a == b:...
I have a function (and a constructor) that should be able to take integer and floating point values. In fact I want it to take an int64_t or a long double, so what I want is, class Foo { public: Foo(int64_t value=0); Foo(long double value); }; However if I do this and try Foo f = 1; the compiler complains about the convers...