I am having trouble believing the following code is the most efficient way to move a value from the stack to ST(0): .data var dd 4.2 tmp dd ? .code mov EAX, var push EAX ; top of stack now contains a value ; move it to ST(0) pop EAX mov tmp, EAX fld tmp Is the temporary variable really necessary? Further, is there an easier way to g...
Hello, Hoping to get some feedback from someone more experienced here. I haven't dealt with the dreaded floating-point calculation before... Is my understanding correct that with Ruby BigDecimal types (even with varying precision and scale lengths) should calculate accurately or should I anticipate floating point shenanigans? All my va...
I found this code in NVIDIA's CUDA SDK samples. void computeGold( float* reference, float* idata, const unsigned int len) { reference[0] = 0; double total_sum = 0; unsigned int i; for( i = 1; i < len; ++i) { total_sum += idata[i-1]; reference[i] = idata[i-1] + reference[i-1]; } // Here it sho...
I have a piece of code that behaves differently depending on whether I go through a dictionary to get conversion factors or whether I use them directly. The following piece of code will print 1.0 == 1.0 -> False But if you replace factors[units_from] with 10.0 and factors[units_to ] with 1.0 / 2.54 it will print 1.0 == 1.0 -> True #!...
I am facing an issue while dividing a double with an int. Code snippet is : double db = 10; int fac = 100; double res = db / fac; The value of res is 0.10000000000000001 instead of 0.10. Does anyone know what is the reason for this? I am using cc to compile the code. ...
how can i create method which return sqrt of given nunber? for example sqrt(16) returns 4 and sqrt(5) returns 2.3... please help i am using java and know Math.sqrt() API function but i need method itself ...
For a simple utility I'm working on, I need a script that converts a given decimal value to a 32bit floating-point hexadecimal value. For example, I know 1 is 3F800000 and 100 is 42C80000, however I don't know how to return these results with any number. If somebody knows a simple formula or even a complex way to go about doing this, ple...
If have the following C function, used to determine if one number is a multiple of another to an arbirary tolerance #include <math.h> #define TOLERANCE 0.0001 int IsMultipleOf(double x,double mod) { return(fabs(fmod(x, mod)) < TOLERANCE); } It works fine, but profiling shows it to be very slow, to the extent that it has become a...
I have noticed an interesting behavior with float rounding / truncation by the C# compiler. Namely, when a float literal is beyond the guaranteed representable range (7 decimal digits), then a) explicitly casting a float result to float (a semantically unnecessary operation) and b) storing intermediate calculation results in a local vari...
What's the easiest way to convert a floating point number to its binary representation in Javascript? (e.g. 1.0 -> 0x3F800000). I have tried to do it manually, and this works to some extent (with usual numbers), but it fails for very big or very small numbers (no range checking) and for special cases (NaN, infinity, etc.): function flo...
I have a method that deals with some geographic coordinates in .NET, and I have a struct that stores a coordinate pair such that if 256 is passed in for one of the coordinates, it becomes 0. However, in one particular instance a value of approximately 255.99999998 is calculated, and thus stored in the struct. When it's printed in ToStrin...
What are some good do-s and don't-s for floating point arithmetic (IEEE754 in case there's confusion) to ensure good numerical stability and high accuracy in your results? I know a few like don't subtract quantities of similar magnitude, but I'm curious what other good rules are out there. ...
Let's say I have some code that does some floating point arithmetic and stores the values in doubles. Because some values can't be represented perfectly in binary, how do I test for equality to a reasonable degree of certainty? How do I determine what "reasonable" means? Can double.Epsilon be used in some way? Update Couple things....
I'm currently measuring the performance of some code in FLOPS. This code presents some arithmetic negate instructions like this one: d = -a where d and a are floating point variables. The architecture I'm currently using does have specific negate instructions. Should I have to take into account this kind of operations to measure FLOPS...
Possible Duplicate: What represents a double in sql server? Is there an exact equivalent for the .NET Double type in SQL Server? Failing that, is there one which gives a good approximation? EDIT: It looks like this link gives the most definitive answer: Mapping CLR Parameter Data ...
An user must enter a number into a mask-edit control. But that number is, depending on the underlying property, one of any .NET numeric type. For instance. If the property is sbyte, the maximum number the user can enter (with any digit, from 0 to 9) is 99 and the minimum is -99 (because sbyte ranges from -128 to 127). That is something ...
hello. I have ran into curious problem. Algorithm I working on consist of lots of computations like this q = x(0)*y(0)*z(0) + x(1)*y(1)*z(1) + ... where the length of summation is between 4 and 7 The original computations are all done using 64-bit precision. For experimentation, I try using 32-bit precision for x,y,z input values (...
I want to compare two floats in php, below is a sample code $a = 0.17; $b = 0.17; if($a == $b ){ echo 'a and b are same'; } else { echo 'a and b are not same'; } In this code it return result of else condition instead of if condition ,even $a and $ b are same. Is there any special way to handle/compare float in php. If yes then ple...
Hey Everyone! I'm currently messing around in FORTRAN 77 and I've ran into a problem that I can't seem to figure out. I'm trying to read from a file that looks similar to below: 000120 Description(s) here 18 7 10.15 000176 Description(s) here 65 20 56.95 ... The last column in each row is a monetary...
I'm have access to a 3rd party application's database, and I see a field called "date" which stores date/time values as floating point numbers, but I'm not sure how this floating point number is mapped to a date/time. There is no documentation for this database. Here is some sample data: date-field actual-date-time 253507382.168...