Currently i have this method: static boolean checkDecimalPlaces(double d, int decimalPlaces){ if (d==0) return true; double multiplier = Math.pow(10, decimalPlaces); double check = d * multiplier; check = Math.round(check); check = check/multiplier; return (d==check); } But this method fails for chec...
Hello, I'm using java and referring to the "double" datatype. To keep it short, I'm reading some values from standard input that I read in my code as doubles (I would much rather use something like BigInteger but right now it's not possible). I expect to get double values from the user but sometimes they might input things like: 999999...
Which libraries for Java are there that have a fast implementation for floating point or fixed point operations with a precision of several thousands of digits? How performant are they? A requirement for me is that it implements a multiplication algorithm that is better than the naive multiplication algorithm that takes 4 times as much ...
Consider the following signature in C#: double Divide(int numerator, int denominator); Is there a performance difference between the following implementations? return (double)numerator / denominator; return numerator / (double)denominator; return (double)numerator / (double)denominator; I'm assuming that both of the above return ...
I seem to be losing a lot of precision with floats. For example I need to solve a matrix: 4.0x -2.0y 1.0z =11.0 1.0x +5.0y -3.0z =-6.0 2.0x +2.0y +5.0z =7.0 this is the code i use to import the matrix from a text file: f = open('gauss.dat') lines = f.readlines() f.close() j=0 for line in lines: bits = string.split(line, ',') ...
I'm working with some number crunching code that, by its nature, is floating-point intensive and and just plain slow. It's research code, so it can be tailored to one architecture, and is running on a Core 2 Quad box. My understanding is that, for the Pentium 4/Netburst architecture, Intel severely stripped down the x87 FPU, and adopte...
I have an application that was developed for Linux x86 32 bits, there are lots of floating-point operations and a lot of tests depending on the results. Now we are porting it to x86_64 but the test results are different in these architecture. We don't want to keep a separate set of results for each arch. According to this article the pr...
Hi, I am working on a calculator that allows you to perform calculations past the decimal point in octal, hexadecimal, binary, and of course decimal. I am having trouble though finding a way to convert floating point decimal numbers to floating point hexadecimal, octal, binary and vice versa. The plan is to do all the math in decimal a...
So, title says it all. Why are floating point values so prolific in computer programming. Due to problems like rounding errors, and not being able to even accurately represent numbers such as 0.1, I really can't see how they got as far as they did. I understand that the computation is faster with floating point numbers, however, I ...
So far I've seen many posts dealing with equality of floating point numbers. The standard answer to a question like "how should we decide if x and y are equal?" is abs(x - y) < epsilon where epsilon is a fixed, small constant. This is because the "operands" x and y are often the results of some computation where a rounding error is in...
I understand that floating point calculations have accuracy issues and there are plenty of questions explaining why. My question is if I run the same calculation twice, can I always rely on it to produce the same result? What factors might affect this? Time between calculations? Current state of the CPU? Different hardware? Language ...
I want to truncate the decimals like below i.e. 2.22939393 -> 2.229 2.22977777 -> 2.229 ...
I have .NET app with a floating point value that I want to adjust up and down by a user adjustable increment with something like a spinner control. Before I go off and build my own, does something like this already exist? ...
In our app, we currently live with the legacy of a decision to store all engineering data in our database in SI. I worry that we may run the risk of not having sufficient precision and accuracy in our database or in .NET numeric types. I am also worried that we may see artifacts of floating-point maths (although that is probably a quest...
I have a program that needs to do a *compile time checkable** map from one known set of values to another known set of values: in out ------------ 8 37 10 61 12 92 13 1/4 109 15 1/4 151 etc This would be easy if the inputs were either integers or evenly spaced. I'm going to be iterating over the rows but also w...
Hi, I am doing some float manipulation and end up with the following numbers: -0.5 -0.4 -0.3000000000000000004 -0.2000000000000000004 -0.1000000000000000003 1.10E-16 0.1 0.2 0.30000000000000000004 0.4 0.5 The algorithm is the following: var inc:Number = nextMultiple(min, stepSize); trace(String(inc)); private function nextMultiple(...
related to: comparing ieee floats and doubles for equality Should we compare floating point numbers for equality against a relative error Most effective way for float and double comparison However with regard to 80 bit IEEE floats (see section 8.2) on an x86 In particular I like the this implementation using a count of representable...
i wrote a class that tests for equality, less than, and greater than with two doubles in java. My general case is comparing price that can have an accuracy of a half cent. 59.005 compared to 59.395. Is the epsilon i chose adequate for those cases? thanks... private final static double EPSILON = 0.00001; /** * Returns true if two dou...
Make a new AS3 Document in Flash, paste in the following code and run it: var a:Number=0; trace(a) // 0 a+=0.3; trace(a) // 0.3 a+=0.3; trace(a) // 0.6 a+=0.3; trace(a) // 0.8999999999999999 a+=0.3; trace(a) // 1.2 a+=0.3; trace(a) // 1.5 a+=0.3; trace(a) // 1.8 a+=0.3; trace(a) // 2.1 a+=0.3; // ^ This is th...
Following on from this question, what would be the best way to represent a System.Decimal object in a Protocol Buffer? ...