I'm having a terrible time convincing myself what I've done here is a good idea. The specific section I find objectionable is: return ((float)($now+$sec).'.'.$mic); In order to preserve the floating point precision, I'm forced to either fall back on the BC or GMP libraries (neither of which is always available). In this case, I've res...
Hello. I'm trying to convert a single precision floating point number into an ascii representation of the four bytes that make it (sign/exponent/mantissa). My current code is: Integer.toString(Float.floatToRawIntBits(f),16); Float.floatToRawIntBits makes an integer using the same bytes, and then Integer.toString creates the ascii dig...
Possible Duplicate: Is JavaScripts math broken? This seems really stupid, but when running this, it will display 7.300000000000001 instead of 7.3 <script language="javascript"> function buttonClicked() { var mySum = 1.1 + 1.1 + 1.1 + 0 + 4; alert(mySum); } </script> What would cause this? I have th...
Is it a good idea to use IEEE754 floating point NaN (not-a-number) for values which are undefined for non-mathematical reasons? In our case they are not yet set because the values have not been received from some other device. The context is an embedded system using IEC1131 REAL32 values. Edit: The programming language is C, so we would...
Hi I have a float variable and would like to get only the part after the comma, so if I have 3.14. I would like to get 14 as an integer. How can I do that? ...
This is one of those silly intro questions, but how does one convert from an int or a decimal to a float in C#. I need to use a float for a third-party control but I don't use them in my code and I'm not sure how to end up with a float. ...
The MSDN documentation mentions that double type includes negative zero. However, both -1.0 / double.PositiveInfinity and -double.Epsilon / 2 appear to return normal 0 (and compare equal to it). How can I get -0? ...
How to convert hexadecimal string to single precision floating point in Java? For example, how to implement: float f = HexStringToFloat("BF800000"); // f should now contain -1.0 I ask this because I have tried: float f = (float)(-1.0); String s = String.format("%08x", Float.floatToRawIntBits(f)); f = Float.intBitsToFloat(Integer.valu...
I did some testing with floating point calculations to minimize the precision loss. I stumbled across a phenomen I want to show here and hopefully get an explanation. When I write print 1.0 / (1.0 / 60.0) the result is 60.0024000960 When I write the same formula and do explicit casting to float print cast(1.0 as float) / (cast(1...
There have been several questions posted to SO about floating-point representation. For example, the decimal number 0.1 doesn't have an exact binary representation, so it's dangerous to use the == operator to compare it to another floating-point number. I understand the principles behind floating-point representation. What I don't under...
Is there a difference in double size when I run my app on 32 and 64 bit environment? If I am not mistaken the double in 32 bit environment will take up 16 digits after 0, whereas the double in 64 bit will take up 32 bit, am I right? ...
I have the following simple program that multiplies two different floating point numbers many times. As you can see, one of the numbers is very small. When I calculate the time of executing both multiplications, I was surprised that the little number takes much longer than the other one. It seems that working with small doubles is slower...
How are floating points represented and interpreted by a compiler. I am trying to understand that so I can easily interpret what byte array would mean for floats and doubles. Thanks ...
A decade or two ago, it was worthwhile to write numerical code to avoid using multiplies and divides and use addition and subtraction instead. A good example is using forward differences to evaluate a polynomial curve instead of computing the polynomial directly. Is this still the case, or have modern computer architectures advanced t...
How many distinct numbers are from 1.5 x 10-45 to 3.4 x 1038 (IEE754 single precision floats)? ...
I am seeing very, VERY strange behavior when I run certain reports: >> p = BigDecimal.new('0.1785990254E5') => #<BigDecimal:b649b978,'0.1785990254E5',16(16)> >> q = BigDecimal.new('0.76149149E4') => #<BigDecimal:b64968d8,'0.76149149E4',8(16)> >> p-q => #<BigDecimal:b6495ab4,'0.124498764E5',16(32)> >> p.to_s => "17859.90254" >> q.to_s =>...
I'm storing values in a column defined as float. When I read back these values from the database they sometimes differ much more from the original values than I would expect (I know that float doesn't store exact values, please take a look at the example to see what I mean). Here's my test case: drop table if exists float_test; create...
I'm doing some work involving MIPS assembly, and I keep coming across these four floating-point load/store pseudoinstructions: l.s, l.d, s.s, s.d. I found some documentation online and figured out that there are four "actual" instructions that seem to do the same thing: lwc1, ldc1, swc1, and sdc1. My only question is, what's the differe...
Is there a good introductory level documentation on floating point related practical pitfalls? I need to give some reading material on this subject to a fairly bright CS student. I remember seeing a good comprehensive manual somewhere couple of years ago, but now, when I need it, I can't find one. ...
I'm using the following piece of code and under some mysterious circumstances the result of the addition is not as it's supposed to be: double _west = 9.482935905456543; double _off = 0.00000093248155508263153; double _lon = _west + _off; // check for the expected result Debug.Assert(_lon == 9.4829368379380981); // sometimes i get 9.48...