views:

515

answers:

4

What would be the correct way of converting color value from float to byte? At first I thought b=f*255.0 should do it, but now I'm thinking, that in this case only the exact 1.0 will be converted to 255, but 0.9999 will already be 254 which is probably not what I want...

It seems that b=f*256.0 would be better except that it would have an unwanted case of making 256 in the case of exact 1.0.

In the end I'm using this:

#define F2B(f) ((f) >= 1.0 ? 255 : (int)((f)*256.0))
A: 

I believe the correct is floor(f*256), not round. This will map the interval 0..1 to exactly 256 zones of equal length.

[EDIT] and check 256 as a special case.

Pavel Radzivilovsky
OK, but then what about 1.0*265=256?
inkredibl
floor(clamp(f, 0, 0.9999999)*256)
Martin
+5  A: 

1.0 is the only case that can go wrong, so handle that case separately:

b = floor(f == 1.0 ? 255 : f * 256.0)

Also, it might be worth forcing that f really is 0<=f<=1 to avoid incorrect behaviour due to rounding errors (eg. f=1.0000001).

f2 = max(0.0, min(1.0, f))
b = floor(f2 == 1.0 ? 255 : f2 * 256.0)
Mark Byers
The only thing that concerns me is that now 255 suddenly has a subtly higher range than all the others ;).
inkredibl
The "problem" comes from having a closed interval instead of a half-closed interval. There's no way to fix this without having one interval slightly larger than the others. Console yourself by knowing that the distribution of floats in the interval [0,1] is not uniform (they are more densely packed near zero) so there's no guarantee that the other intervals are the same size either.
Mark Byers
255 should cover values from 0.99609375 included to 1.0 excluded. This answer suggests to include 1.0 to the interval. Indeed this is very subtle. For me this is the best possible answer.
mouviciel
To further clarify: if you start with a closed interval and split it into (say) two intervals, the only thing you can do is to make one half-closed interval and one closed interval. There's no way around the fact that one interval is "slightly larger" than the other. It's best not to worry about it. :)
Mark Byers
Note: I updated my comment to explicitly (rather than implicitly) call the floor function, in case it is unclear.
Mark Byers
Well, it seems it's as good as it can get.
inkredibl
+1  A: 

As the simple:

int value = (int)(f*255);

isn't giving you the result you require, any other linear conversion probably isn't going to cut it either.

What you'll need is a non-linear function something that gives you more values at the extremes and compresses the middle range. There are probably spline functions you could use - I'll need to do more research.

Notes on the simple conversion

This will convert 1.0 to 255 (barring rounding error) and 0.9999 to 254.

If rounding error on the 1.0 case is an issue then handle it explicitly.

The only issue might be the resolution. 1.0 - (255.0 / 254.0) returns 0.00392. Is this good enough?

ChrisF
Well I've tried exactly that in the beginning, but as I mentioned - it doesn't seem right, that white has almost no chance at all ;).
inkredibl
@inkredibl - Missed that! However, back when I was doing 3d graphics we used this to do the conversion (and it's inverse) without any problems or noticeable artefacts.
ChrisF
+1  A: 

Why not try something like

b=f*255.999

Gets rid of the special case f==1 but 0.999 is still 255

ammoQ
Agree, it's simpler in code but has less precision than the accepted solution.
inkredibl
inkredibl: You will be hard-pressed to find examples where the difference really matters...
ammoQ