views:

140

answers:

2

I would like to implement a frac function in C# (just like the one in hsl here http://msdn.microsoft.com/en-us/library/bb509603%28VS.85%29.aspx) but since it is for a very processor intensive application i would like the best version possible. I was using something like

public float Frac(float value)
{
   return value - (float)Math.Truncate(value);
}

but I'm having precision problems, for example for 2.6f it's returning in the unit test

Expected: 0.600000024f But was: 0.599999905f

I know that I can convert to decimal the value and then at the end convert to float to obtain the correct result something like this:

public float Frac(float value)
{
   return (float)((decimal)value - Decimal.Truncate((decimal)value));
}

But I wonder if there is a better way without resorting to decimals...

+2  A: 

float (an alias for System.Single) is only precise to 7 digits. You shouldn't really compare floating point values using equality anyway. You should instead check that they are within a tolerable range - for example that the difference between the expected and actual values is not more than 0.000001.

Assert.AreEqual(expected, actual, delta);

If you use double instead, you will be precise to 15-16 digits.

Josh Einstein
I do not know if he needs that, but, in that case, he can't tell the difference between 2.5999999999 and 2.6 and 2.600000001 etc... Decimal should be the way to go here.
Recep
When dealing with true floating point types (which decimal is not), there *is* no meaningful difference between 2.5999999999 and 2.6 if the difference exceeds the precision allowed by the type. Using decimal is slower and has a much higher storage cost that may not be needed if the extra precision is not necessary.
Josh Einstein
My problem is precisely that i don't need the extra precision. I've seen posts that resort to tricks with string's that can achieve the same result however i fell that is not the fastest solution. The best way till now is decimal. I was wondering only if there is some clever trick with maths that could make this faster in c#
Ivo Leitão
The problem (in my opinion) with floating point types is when displayed, values like 2.5999999999 feel "wrong" to us but in practice (for example as a WPF coordinate) there's no difference between that and 2.6. If you were to display it with a format string within its range of precision it would look fine. In the end I would suggest using double which would avoid the conversion and would display as expected.
Josh Einstein
Unfortunately the problem is not resolved with doubles. I only get a bigger number. For example for 2.6f in the value variablewith the expression (double)value - Math.Truncate((double)value)I get:0.59999990463256836The problem is the same
Ivo Leitão
You shouldn't see the issue if you use double's throughout. I tested it in LINQPad and it produced the expected value.
Josh Einstein
A: 

You are going to need to use decimals as in:

public decimal Frac(decimal value) { return value - Math.Truncate(value); }

So, when you pass 2.6m you will always get .6

Recep
I think i will use decimals for now. At least until a find a cleaver and fastest solution...
Ivo Leitão
I ended up using (float)((decimal)value % 1.0m) which I think is still slow but at least works
Ivo Leitão