views:

10

answers:

1

When I run a query through DBslayer http://code.nytimes.com/projects/dbslayer the floating point results are truncated to a total of six digits (plus decimal point and negative sign when needed).

{ ... "lat":52.2228,"lng":-2.19906, ... }

When I run the same query in MySQL, the results are as expected.

| 52.22280884 | -2.19906425 |

Firstly, am I correct in identifying DBSlayer as the cause of this effect? (Or the JSON library it uses, etc.)

Secondly, is this floating point precision configurable within DBSlayer?

Thanks,

Chris.

P.S. Ubuntu 9.10, x86_64

Path: . URL: http://dbslayer.googlecode.com/svn/trunk Repository Root: http://dbslayer.googlecode.com/svn Repository UUID: 5df2be84-4748-0410-afd4-f777a056bd0c Revision: 65 Node Kind: directory Schedule: normal Last Changed Author: dgottfrid Last Changed Rev: 65 Last Changed Date: 2008-03-28 22:52:46 +0000 (Fri, 28 Mar 2008)

A: 

I eventually solved this problem.

There are several places where DBSlayer uses printf's %g to format doubles. This gives only 6 significant digits.

Replacing all of these occurrences with %.15g, and recompiling, has solved the problem.

chrisdew