I don't believe so. The effect that each of the primary colors has on the eye is not identical. I remember reading something about this many years ago.
Basically, I think that red has the most effect on the receptors in the retina so, to get a true average in terms of effect, you would have to scale the three primary colors based on that.
However, I would question whether that sort of level of detail would be required in the case where there's only 256 possible scales of each primary color. You may find that the simplified averaging you're proposing will be more than adequate.