I'll be the first to admit that my overall knowledge of low level programming is a bit sparse. I understand many of the core concepts but I do not use them on a regular basis. That being said I was absolutely astounded at how much code was needed for dtoa.c. For the past couple months I have been working on an ECMAScript implementation...
I have a warning in my C++ code, regarding the use of FLT_MAX. The code is (very simply): const float a_lot = FLT_MAX; The generated warning is: warning C4756: overflow in constant arithmetic And it doesn't help if I change the code to: const float a_lot = std::numeric_limits<float>::max(); I'm using Microsoft Visual Studio 2...
Hi, I'm trying to write a synthesizable 3D rasterizer in Verilog/SystemVerilog. The rasterizer right now is not really a 3D rasterizer: it just receives six 32-bits floats for vertex position (vertA_pos_x, vertA_pos_y, vertB_pos_x, vertB_pos_y, vertC_pos_x, vertC_pos_y) and nine 8-bits integers for vertex coloring (vertA_color_r, vertA_...
I want to convert a floating point user input into its integer equivalent. I could do this by accepting an input string, say "-1.234" and then I could just explicitly convert each character to it's decimal representation. (big endian by the way). So I would just say for the example I gave, -1.234 = 1|01111111|00111011111001110110110 ...
Hi, I am thinking recently on how floating point math works on computers and is hard for me understand all the tecnicals details behind the formulas. I would need to understand the basics of addition, subtraction, multiplication, division and remainder. With these I will be able to make trig functions and formulas. I can guess something...
We have recently purchased a dual Intel X5650 workstation to run a floating-point intensive simulation, under Ubuntu 10.04. Each X5650 has 6 cores, so there are 12 cores in total. The code is trivially parallel, so I have been running it mostly with 12 threads, and observing approximately "1200%" processor utilization through "top". Hy...
I was reading today about researchers discovering that NVidia's Phys-X libraries use x87 FP vs. SSE2. Obviously this will be suboptimal for parallel datasets where speed trumps precision. However, the article author goes on to quote: Intel started discouraging the use of x87 with the introduction of the P4 in late 2000. AMD deprecate...
Possible Duplicate: How to format a decimal How can I limit my decimal number so I'll get only 3 digits after the point? e.g 2.774 ...
First of all, this is not a floating point newbie question. I know results of floating point arithmetic (not to mention transcendental functions) usually cannot be represented exactly, and that most terminating decimals cannot be represented exactly as binary floating point numbers. That said, each possible floating point value correspo...
Is there any way to check if a long integer is too large to convert to a float in python? ...
hello. What is a good way to bin float precision numbers? something like this, but perhaps more efficient? x = 1; for i = 0,size-1 { // loop over number of bins if (value > x) { ++bin[i]; return; } x *= 0.1; } ++bin[size-1]; // increment last bins I have thought of getting exponent directly, frexp, and using that...
I am writing a piece of code in which i have to convert from double to float values. I am using boost::numeric_cast to do this conversion which will alert me of any overflow/underflow. However i am also interested in knowing if that conversion resulted in some precision loss or not. For example double source = 1988.1012; floa...
I was reading that you have to be careful with floating point comparisons and calculations in PHP. I'm not an expert at PHP so I thought I would ask the community what is the best way to do accurate decimal calculations in PHP. I'm developing a timecard application where it calculates all the time entered for the week. Anything over 40 ...
I just read this nice answer given on how to compare floating-point values for equality. The following (slightly modified by me) is suggested instead of straight-forward comparison to 0: const double epsilon = 1e-5; double d = ...; if (Math.Abs(d) < epsilon) { // d is considered equal to 0. } My question is about the name of the ...
Does GCC have an equivalent compiler switch to VC's floating point model switch (/fp)? In particular, my application benefits from compiling with /fp:fast and precision is not a big deal, how should I compile it with GCC? ...
please ignore this post, I misread algorithm, so the question is not relevant. However, I cannot close post anymore. Please vote to close I have been using certain algorithm from numerical recipes, which converges to zero via underflow: // all types are the same floating type sum = 0 for (i in 0,N) sum += abs(V[i]); my question, h...
Hi, I was trying to understand the floating point representation in C using this code (both float and int are 4 bytes on my machine): int x = 3; float y = *(float*) &x; printf("%d %e \n", x, y); We know that the binary representation of x will be the following 00000000000000000000000000000011 Therefore I would have expected y to be...
The struct module is useful when you're trying to convert data to and from binary formats. However, recently I came across a file format specification that uses the binary16 floating point format. I looked through the Python documentation, but can't find anything that can convert to and from it. What would be the best way to convert this...
This is a follow-up to Testing for floating-point value equality: Is there a standard name for the “precision” constant?. There is a very similar question Double.Epsilon for equality, greater than, less than, less than or equal to, greater than or equal to. It is well known that an equality test for two floating-point values x and y s...
Are there any performance differences between float x, y; // Set x and y if(x > y) { // do something } and float x,y; // Set x and y if(x.CompareTo(y) > 0) { // do something } Are they doing the same thing behind the scenes or is there more to it. I have a piece of performance critical code that does this compari...