I'm drawing a color selection button and I'm looking for a nice and simple formula to get a good text color (foreground) for a given background color in RGB.
A simple try would be to just take the complement color but this will produce an odd looking button for colors like pure blue or pure red.
Is there something well known that does this?
If it matters at all, I'm using QT.
I haven't actually tried this idea, but maybe thinking of the color in a space other than RGB might be the way. For example, convert the RGB value to something like HSB, and flip the hue?
I'm no expert on programming things related to RGB, but from a designer's perspective, often the most readable color will be just a much lighter (if the background color is dark) or darker (if the background color is light) version of the same shade.
Basically you'd take your RGB values and if they're closer to 0 (dark) you'd push them each up by an equal amount for your foreground color, or vice versa if it's a light BG.
Complement colors can actually be really painful on the eyes for readability.
I usually look at color complements, they also have color complement wheels to help
http://www.makart.com/resources/artclass/cwheel.html
If your color is HSL, flip the Hue by 180 degrees for a decent calculation
I thing that converting to HSV might be the way, but IMO changing hue would look weird. I'd try keeping the hue and fiddling with value and maybe saturation (light red buttons with dark red text ... hm sounds scary :-) ).
Color combinations often look terrible when not carefully chosen. Why not use either white or black for the text, depending on the Brightness of the color. (Will need to convert to HSB first.)
Or let the user choose either black or white text.
Or use pre-defined combinations. This is what Google does in their calendar product.
You are better off with a high difference in luminosity. In general, colored backgrounds with colored text suck for readability, hurting the eyes over time. Lightly tinted colors (e.g. in HSB, S~10%, B>90%) with black text work fine, or lightly tinted text over a black background. I'd stay away from coloring both. Dark text (b~30%, s>50%) with a subtle coloration over a white background can also be fine. Yellow (amber) text on a deep blue background has excellent readability, as does amber or green on black. This is why old dumbterms (vt100, vt52, etc.) went for these colors.
If you really need to do color-on-color for the 'look', you could reverse both H and B, while pinning saturation at a moderate to low level.
And one last note: if you have a 50% gray background, rethink your interface. You're robbing yourself of half your dynamic range! You're alienating low-visibility users, including anyone over 35...
For maximum legibility, you want maximum brightness contrast without getting into hues which don't work together. The most consistent way to do this is to stick with black or white for the text color. You might be able to come up with more aesthetically pleasing schemes, but none of them will be more legible.
To pick between black or white, you need to know the brightness of the background. This gets a little more complicated, due to two factors:
The perceived brightness of the individual primaries red, green, and blue are not identical. The quickest advice I can give is to use the traditional formula to convert RGB to gray - R*0.299 + G*0.587 + B*0.114. There are lots of other formulas.
The gamma curve applied to displays makes the middle gray value higher than you'd expect. This is easily solved by using 186 as the middle value rather than 128. Anything less than 186 should use white text, anything greater than 186 should use black text.