i have array of floats where data are stored with varying decimal points so some are 123.40000, 123.45000, 123.45600...now if i want to print these values in the string without the 0s in the end in printf() so that they are 123.4, 123.45, 123.456, without those 0s in the end. is this possible? if so how? ...
I have a column that is currently a floating-point number and I need to check if all the values in the column are integers. What's the easiest way to do this? ...
Are there any situations where it would make more sense to use a single datatype instead of a double? From my searching, the disadvantage to a double is that it requires more space, which isn't a problem for most applications. In that case, should all floating point numbers be doubles? A little background info: I'm working with an app...
I'm writing a set of numeric type conversion functions for a database engine, and I'm concerned about the behavior of converting large integral floating-point values to integer types with greater precision. Take for example converting a 32-bit int to a 32-bit single-precision float. The 23-bit significand of the float yields about 7 dec...
Why does 0.1 + 0.1 + 0.1 - 0.3 evaluates to 5.5511151231257827e-17 in Python? ...
So I know how to print a floating point number with a certain decimal places. My question is how to return it with a specified number of decimal places? Thanks. ...
This is in reference to the comments in this question: This code in Java produces 12.100000000000001 and this is using 64-bit doubles which can present 12.1 exactly. – Pyrolistical Is this true? I felt that since a floating point number is represented as a sum of powers of two, you cannot represent 12.1 exactly, no matter how many...
float totalAmount = 0; . . .//totalAmount assigned value 1.05 correctly . totalAmount += float.Parse(dataRow["Amt"].ToString()); //where dataRow["Amt"] has value 4.93 The answer I get for totalAmount is 5.97999954 instead of 5.98 Why is this happening? ...
Is it possible to set the display precision of a float in Ruby? Something like: z = 1/3 z.to_s #=> 0.33333333333333 z.to_s(3) #=> 0.333 z.to_s(5) #=> 0.33333 Or do I have to override the to_s method of Float? ...
I am converting double values to string like this: std::string conv(double x) { char buf[30]; sprintf(buf, "%.20g", x); return buf; } I have hardcoded the buffer size to 30, but am not sure if this is large enough for all cases. How can I find out the maximum buffer size I need? Does the precision get higher (and theref...
First my context is that of a compiler writer who needs to convert floating point literals (strings) into float/double values. I haven't done any floating point programming the last 15 years so i'm pretty sure this is a total stupid newbie question. double res; errno=0; *res = strtod((const char*)literal,NULL); if (errno==ERANGE) thr...
I'm trying to construct an algorithm that validates that a double value is a member of a range defined with min, max and step values. The problem is checking that the value obeys the step rule. For integers this can be easily done: boolean validate(int value, int min, int step, int max){ return value >= min && value ...
I'm writing a 2D space RTS game in C#. Single player works. Now I want to add some multiplayer functionality. I googled for it and it seems there is only one way to have thousands of units continuously moving without a powerful net connection: send only the commands through the network while running the same simulation at every player. ...
I understand due to the inexact representation of floating points, the following code 'feels' inconsistent. "%.1f" % 1.14 # => 1.1 "%.1f" % 1.15 # => 1.1 "%.1f" % 1.16 # => 1.2 "%.0f" % 1.4 # => 1 "%.0f" % 1.5 # => 2 "%.0f" % 1.6 # => 2 However, is there an easy way of doing consistent floating points rounding by 5? One way might be t...
I am trying to setup a static assert (outside the main function) with GCC v4.3.x: #define STATIC_ASSERT(cond) extern void static_assert(int arg[(cond) ? 1 : -1]) STATIC_ASSERT( (double)1 == (double)1 ); // failed but when I use float numbers, the assert always failed. Is it possible to run this static assert properly ? ...
This is more of a question of curiosity but does anyone know how negative precision values are handled in C++? For example: double pi = 3.14159265; cout.precision(-10); cout.setf(ios::fixed, ios::floatfield); cout << pi << endl; I've tried this out and using GCC and it seems that the precision value is ignored but I was curious if ...
Why Does -22/10 return -3 in python. Any pointers regarding this will be helpful for me. ...
The question is pretty "simple". I have an engine that can work with both DirectX9 and DirectX10 with SlimDX in C#. I use the Stopwatch class to time the game and test speed of some routine. Strangely exactly after the creation of the DirectX9 the floating point of the StopWatch go crazy and the GetTimeStamp()/Frequency division returns ...
I am having some trouble with IEEE floating point rules preventing compiler optimizations that seem obvious. For example, char foo(float x) { if (x == x) return 1; else return 0; } cannot be optimized to just return 1 because NaN == NaN is false. Okay, fine, I guess. However, I want to write such that the ...
If I apply Binet Formula and Recursive formula for finding the fibonaci series, there is a discrepancy in result. Why? Basically I am a student and it is our assignment to implement the fibbonacci series. So I while making the experiment I came across this situation. Thanks in advance ...