Im currently reading Code Complete by Steve McConnell, specifically page 295 on floating-point numbers. When i ran the following code: double nominal = 1.0; double sum = 0.0; for (int i = 0; i < 10; i++) { sum += 0.1; Console.WriteLine("sum: " + sum.ToString()); } ...
float f = 5.13; double d = 5.13; float fp = f - (float)Math.floor(f); double dp = d - Math.floor(d); Isn't there any faster way than calling an external function every time? ...
I am interested to learn about the binary format for a single or a double type used by C++ on Intel based systems. I have avoided the use of floating point numbers in cases where the data needs to potentially be read or written by another system (i.e. files or networking). I do realise that I could use fixed point numbers instead, and ...
Guys, I want to know if float variables can be used in sprintf() function. Like, if we write: sprintf(str,"adc_read = %d \n",adc_read); where adc_read is an integer variable, it will store the string "adc_read = 1023 \n" in str (assuming that adc_read = 1023) How can I use a float variable in place of integer? ...
I have a primitive float and I need as a primitive double. Simply casting the float to double gives me weird extra precision. For example: float temp = 14009.35F; System.out.println(Float.toString(temp)); // Prints 14009.35 System.out.println(Double.toString((double)temp)); // Prints 14009.349609375 However, if instead of casting, I o...
I'd like to know if there is a way to improve the accuracy of calculating a slope. (This came up a few months back here). It seems by changing: float get_slope(float dXa, float dXb, float dYa, float dYb) { return (dXa - dXb)/(dYa - dYb); } to float get_slope(float dXa, float dXb, float dYa, float dYb) { return dXa/(dYa - d...
Why does this code sometimes return 1E+1 whilst for other inputs (e.g. 17) the output is not printed in scientific notation? BigDecimal bigDecimal = BigDecimal.valueOf(doubleValue).multiply(BigDecimal.valueOf(100d)).stripTrailingZeros(); System.out.println("value: " + bigDecimal); ...
Hi there. I'm writing an app that revolves around getting sets of numerical data from a file. However, since the data is acquired in string form, I have to convert it to floats, which is where the fun starts. The relevant section of my code is as shown (lines 65-73): ft = [] puts "File Name: #{ARGV[0]}" File.open(ARGV[0], "r...
I work daily with Python 2.4 at my company. I used the versatile logarithm function 'log' from the standard math library, and when I entered log(2**31, 2) it returned 31.000000000000004, which struck me as a bit odd. I did the same thing with other powers of 2, and it worked perfectly. I ran 'log10(2**31) / log10(2)' and I got a round 3...
Given Wikipedia's article on Radix Point, how would one calculate the binary equivalent of 10.1 or the hex equivalent of 17.17? For the former, what is the binary equivalent of a tenth? For the latter, the hex representation of 17/100? I'm looking more for an algorithm than for solutions to just those two examples. ...
I have two matrices x and y, both are results from different algorithms/routines that are supposed to calculate the same result. While I know that the isequal() would check if x and y are the same matrix, the entries in those matrices would not be exactly the same (i.e. some entries may be with 5% off in worst case scenario). In this sce...
I'm using the Statistics::Descriptive library in Perl to calculate frequency distributions and coming up against a floating point rounding error problem. I pass in two values, 0.205 and 0.205, (taken from other numbers and sprintf'd to those) to the stats module and ask it to calculate the frequency distribution but it's getting stuck i...
How come when I multiply 1.265 by 10000 I Get this 126499.99999999999 when using Javascript ...
Having already read this question I'm reasonably certain that a given process using floating point arithmatic with the same input (on the same hardware, compiled with the same compiler) should be deterministic. I'm looking at a case where this isn't true and trying to determine what could have caused this. I've compiled an executable an...
Say you're writing a C++ application doing lots of floating point arithmetic. Say this application needs to be portable accross a reasonable range of hardware and OS platforms (say 32 and 64 bits hardware, Windows and Linux both in 32 and 64 bits flavors...). How would you make sure that your floating point arithmetic is the same on all...
Does anybody know what SAS format is needed in order to convert this string correctly? data _null_; x="0.14553821459"; y=input(x,best32.); put y=; run; ...
Hi, I have a simple web app , and want to save some numbers of Float or Double format in SQL server. but there is a problem , when I try to save 123.66 , In Table I see 123.6600003662109 stored. How nd why my float number changed when save on DB? how can I fix this error? Thanks ...
How can we use them in our codes, and what will cause NaN(not a number)? ...
Is it (easily) possible to use software floating point on i386 linux without incurring the expense of trapping into the kernel on each call? I've tried -msoft-float, but it seems the normal (ubuntu) C libraries don't have a FP library included: $ gcc -m32 -msoft-float -lm -o test test.c /tmp/cc8RXn8F.o: In function `main': test.c:(.text...
I'm trying to calculate the sum of an array of decimal values in PHP, but for some reason it keeps rounding to integers. for example: $oldArray = array(0.00,1000.11,988.92,978.22,964.01,953.07,948.82,917.26,902.56,913.21,904.08,898.86,892.79); $myVar = 0.0; for($k=1;$k<10;$k++) { $myVar += $oldArray[$k]; } print_r($myVar); $oldArray ...