views:

199

answers:

2

I'm building a binary classification tree using mutual information gain as the splitting function. But since the training data is skewed toward a few classes, it is advisable to weight each training example by the inverse class frequency.

How do I weight the training data? When calculating the probabilities to estimate the entropy, do I take weighted averages?

EDIT: I'd like an expression for entropy with the weights.

+1  A: 
Robert Harvey
Yes, I realized this. I was hoping for a weighted version of entropy. I use various entropy estimates to calculate a scores similar to mutual information.
Jacob
A: 

State-value weighted entropy as a measure of investment risk.
http://www56.homepage.villanova.edu/david.nawrocki/State%20Weighted%20Entropy%20Nawrocki%20Harding.pdf

Robert Harvey