Floating point precision in FIT tests

Question

Are there any good/best practices for dealing with floating point values when checking results with the fit framework?

Our application does many calculations. In most situations checking values a precision of more than 6 or 8 digits does not make any sense. So I would like to specify the precision that are used when comparing expected and actual values.

My brute force solution would be to set a precision value and then compare values in my ColumnFixture containing the test cases.

This would result in something like:

!|info.fitnesse.fixturegallery.ColumnFixtureTest| |firstSummand|secondSummand|precision|expectedResult|calculateSumAndCheck()| |2.1|3.3|0.0001|5.3|true|

The major drawback of this approach would be that I would not be able to use the compare features of fit but write my own ones.

Answer 1

You can use consistent string formatting for your floating point types and compare string-to-string. Fit is really doing a ToString() on your values anyway and comparing to the text in your wiki, so if you provide your own string formatting, you do not need to modify the built-in comparisons in Fit.

You didn't specify a language, but in C#, for example, you could use a static helper method or an extension method to the floating point types (float, double, etc.) called something like "ToStringForComparison()". All your test harnesses would need to use this in the property accessors for values you want to compare in your tests.

Edit:

If you want to control the precision, you can do that by passing in the precision value into your formatting function while maintaining a default for values where precision is unimportant.