I'd like to plot mathematical functions and I'm looking for techniques and algorithms for precise and efficient plotting. 2D plotting should suffice. I focus on techniques and algorithms, not libraries or languages. Thanks in advance! ...
Hello, I have the following problems: First: I am trying to do a 32-spaces bitwise left shift on a large number, and for some reason the number is always returned as-is. For example: echo(516103988<<32); // echoes 516103988 Because shifting the bits to the left one space is the equivalent of multiplying by 2, i tried multiplying the...
This might be a repeat, but my google-fu failed to find it. I want to print numbers to a file using the stl with the number of decimal places, rather than overall precision. So, if I do this: int precision = 16; std::vector<double> thePoint(3); thePoint[0] = 86.3671436; thePoint[0] = -334.8866574; thePoint[0] = 24.2814; ofstream file...
The documentation states: "Precision can range from 1 to 38. Scale can range from -84 to 127". How can the scale be larger than the precision? Shouldn't the Scale range from -38 to 38? ...
Hi, Is there a way to set the "minimum" number of decimal places that a std::ostream will output? For example, say I have two unknown double variables that I want to print (values added here for the sake of illustration): double a = 0; double b = 0.123456789; I can set my maximum decimal precision so that I output b exactly std::c...
I am aware of how floating point precision works in the regular cases, but I stumbled on an odd situation in my C# code. Why aren't result1 and result2 the exact same floating point value here? const float A; // Arbitrary value const float B; // Arbitrary value float result1 = (A*B)*dt; float result2 = (A*B); result2 *= dt; F...
I have a machine which uses an NTP client to sync up to internet time so it's system clock should be fairly accurate. I've got an application which I'm developing which logs data in real time, processes it and then passes it on. What I'd like to do now is output that data every N milliseconds aligned with the system clock. So for exampl...
Working on Mac OS X 10.6.2, Intel, with i686-apple-darwin10-g++-4.2.1, and compiling with the -arch x86_64 flag, I just noticed that while... std::numeric_limits<long double>::max_exponent10 = 4932 ...as is expected, when a long double is actually set to a value with exponent greater than 308, it becomes inf--ie in reality it only has...
Hi, I'm using fopen in C to write the output to a text file. The function declaration is (where ARRAY_SIZE has been defined earlier): void create_out_file(char file_name[],long double *z1){ FILE *out; int i; if((out = fopen(file_name, "w+")) == NULL){ fprintf(stderr, "***> Open error on output file %s", file_name)...
Hi,on my 32-bit machine (with an Intel T7700 duo core), I have 15 precision digits for both double and long double types for the C language. I compared the parameters LDBL_DIG for long double and DBL_DIG for double and they are both 15. I got these answers using MVS2008. I was wondering if these results can be compiler dependent or do th...
Hi, I'm printing a variable say z1 which is a 1-D array containing floating point numbers to a text file so that I can import into Matlab or GNUPlot for plotting. I've heard that binary files (.dat) are smaller than .txt files. The definition that I currently use for printing to a .txt file is: void create_out_file(const char *file_nam...
Double has range more than a 64-bit integer, but its precision is less dues to its representation (since double is 64-bit as well, it can't fit more actual values). So, when representing larger integers, you start to lose precision in the integer part. #include <boost/cstdint.hpp> #include <limits> template<typename T, typename TFloat>...
When using the usual formulas to calculate intersection between two 2D segments, ie here, if you round the result to an integer, you get non-symmetric results. That is, sometimes, due to rounding errors, I get that intersection(A,B)!=intersection(B,A). The best solution is to keep working with floats, and compare the results up to a c...
I have a situation where two persons might work on the same order (stored in an MS SQL database) from two different computers. To prevent data loss in the case where one would save his copy of the order first, and then a little later the second would save his copy and overwrite the first, I've added a check against the lastSaved field (d...
I have a situation where I need to find out how many times an int goes into a decimal, but in certain cases, I'm losing precision. Here is the method: public int test(double decimalAmount, int divisor) { return (int) (decimalAmount/ (1d / divisor)); } The problem with this is if I pass in 1.2 as the decimal amount and 5 as the divi...
Hi, I'm very new to Ada and was trying to see if it offers double precision type. I see that we have float and Put( Integer'Image( Float'digits ) ); on my machine gives a value of 6, which is not enough for numerical computations. Does Ada has double and long double types as in C? Thanks a lot... ...
Hello! I wonder how calculators work with precision. For example the value of sin(M_PI) is not exactly zero when computed in double precision: #include <math.h> #include <stdio.h> int main() { double x = sin(M_PI); printf("%.20f\n", x); // 0.00000000000000012246 return 0; } Now I would certainly want to print zero when us...
Hello all, Coding a matrix multiplication in my program, I get precision errors (inaccurate results for large matrices). Here's my code. The current object has data stored in a flattened array, row after row. Other matrix B has data stored in a flattened array, column after column (so I can use pointer arithmetic). protected double[,]...
I have come across a precision issue with double in .NET I thought this only applied to floats but now I see that double is a float. double test = 278.97 - 90.46; Debug.WriteLine(test) //188.51000000000005 //correct answer is 188.51 What is the correct way to handle this? Round? Lop off the unneeded decimal places? ...
I was given this interview question recently: Given a 12-hour analog clock, compute in degree the smaller angle between the hour and minute hands. Be as precise as you can. I'm wondering what's the simplest, most readable, most precise algorithm is. Solution in any language is welcome (but do explain it a bit if you think it's nece...