Floating point calculations with latitudes and longitudes of varying precisions

Question

Background: I receive a long and lat as parameters to a web service. They are typically up to 6 decimal places. When a new request is received, I calculate the distance between the last recorded loc and the long/lat in the params of the request. If the distance is greater than a certain threshold of miles apart, I update the current loc.

Problem: I use the geokit gem/plugin to calculate the distance between the locs. Very rarely, a bug shows up (the zero distance bug mentioned on the author's site - I'm using 1.4.1 which claims the bug is fixed, but I still see it occurring shrug) that causes the distance calc to return something wildly inaccurate when calculating the distance between two points that are identical (this occurs if the user is not moving). This is causing updates to the user's current loc that should not be happening. You're probably wondering - well if it's just updating the loc to be exactly the same coordinates, who cares? Well, the answer is that a bunch of other crap occurs when the loc is updated that makes it an actual issue.

Attempted Solution: I tried to add in logic to manually check if the two locs are identical before calculating the distance and then just skipping the calc and not updating if that is the case. The incoming parameters are long/lats with 6 decimal precision; whereas, in my database, I store the values as floats, which appears to only store 4 decimal places. This is causing my float comparison to always fail and the inappropriate loc updates continue to occur.

Phew, ok so the actual question is: How should I perform this comparison? Should I truncate 2 of the decimal places from the incoming lat/longs, round up somehow so the fourth digit is correct and then compare? Or, should I do a "within a certain range" sort of comparison (e.g. reported_loc.long > current_loc.long - .0001 && reported_loc.long < current_loc.long + .0001)? Also any recommendations for existing ruby gem/plugins or built in functions to do this sort of thing would be much appreciated.

Here is sample output from the log:

[update_loc] Last location history record at lat: 41.5024, long: -81.6816
[update_loc] Current loc at lat: 41.502467, long: -81.681623
[update_loc] Distance from current loc and last loc history: 5795.10615113555 miles
[update_loc] Locs not identical and distance greater than threshold, inserting new loc history
[update_loc] Location update complete

Thank you

Tom

Answer 1

A GPS receiver can give you a location with a precision of many decimals, but that doesn't mean that the measurement is actually that accurate.

Usually about 95% of the measurements lie within a circle of a couple of meters, which is about the same accuracy that you can store with a 32bits float.

However, you will clearly notice rounding errors when you plot a gps log as floats on a map.

Anyway, to do this comparison, I'd use a range instead of rounding the incoming value and comparing that to the database value. You won't be able to detect the smallest movements anymore, but at least you won't get false positives anymore either.

As you're using floats to store stuff, you clearly don't care about millimeters or centimeters anyway.

Answer 2

The usual way to test if two numbers are close is to use abs, ie,

 (reported_loc.long - current_loc.long).abs <= tol

where tol is some pre-specified tolerance, eg, 0.0001.