Rejigging a floating point equation ...

Question

I'd like to know if there is a way to improve the accuracy of calculating a slope. (This came up a few months back here).

It seems by changing:

float get_slope(float dXa, float dXb, float dYa, float dYb) {
    return (dXa - dXb)/(dYa - dYb);
}

to

float get_slope(float dXa, float dXb, float dYa, float dYb) {
    return  dXa/(dYa - dYb) - dXb/(dYa - dYb);
}

might be an improvement. Suggestions?

Edit: It's precision I'm after, not efficiency.

Answer 1

I think you have a typo. You probably mean

return  dXa/(dYa - dYb) - dXb/(dYa - dYb);

I would say that the first form that you gave has higher precision. If dXa and dXb are close and large, then you would lose precision in the two divisions before subtracting.

Answer 2

It really depends on how your the language compiles your code to machine instructions.

The naive view, in which we assume each operation(- and /) is compiled to a machine instruction, would suggest that first function is better because it involves only 3 operations (two subtractions and one division vs three subtractions and two divisions).

Moreover it seems reasonable to assume that division is a more costly operation than subtraction, especially with floats.

One can always pick clever values for a and b such that the slope is always a division of whole numbers.

in response to comment: Picking clever values is easy, assuming calculating the lcm is easy/precise.

int[] get_slope(float Xa, float Xb, float Ya, float Yb) {
    dX = (Xa - Xb)
    dY = (Ya - Yb)
    lcm = (dX,dY)
    int[] slope = [2]
    slope[0] = int(dX*lcm)
    slope[1] = int(dY*lcm)
    return slope
}

slope[0] and slope[1] are both guarantied to be whole numbers, so no precision is lost. The slope array can be treated as a rational number.

Maybe not the answer you're looking for, but an interesting question none the less.

Answer 3

Cast them to double inside the function.

---

Where you're going to run into trouble is when the denominator is near zero, obviously. Your slope will approach infinity. So a lot depends on what you want to do with the slope. Sometimes, if you know your delta y is going to be near zero, you can calculate the reciprocal of slope and use that instead. You can even detect which is smaller--the absolute value of deltax or deltay and return the slope or 1/slope. Also look into atan2().

---

If you know the input is in decimal, and you want the output in decimal as well, you can overcome the loss of precision that is inherent in converting floats to binary and back by doing all calculations with a decimal library. I remember how pleased I was when I used Atari BASIC for decimal calculations, as it used the 6502's BCD mode.

Answer 4

If you don't mind burning some extra cycles you can get better accuracy by doing a loop.

Calculate slope of the line segment between A and B.

Calculate slope of the line segment between {(Xa - (Xa -Xb)), (Ya -(Ya -Yb)} and {(Xb + (Xa - Xb)), (Yb +(Ya - Yb))}... Basically A - slope and B + slope.

Then compare the resulting slopes. If the difference is too high (choose the threshold you want) then keep going, and average all of the slopes at the end.

This can help smooth out anomalies caused by floating point arithmetic for very small slopes.