Estimate gaussian (mixture) density from a set of weighted samples

Question

Assume I have a set of weighted samples, where each samples has a corresponding weight between 0 and 1. I'd like to estimate the parameters of a gaussian mixture distribution that is biased towards the samples with higher weight. In the usual non-weighted case gaussian mixture estimation is done via the EM algorithm. Does anyone know an implementation (any language is ok) that permits passing weights? If not, does anyone know how to modify the algorithm to account for the weights? If not, can some one give me a hint on how to incorporate the weights in the initial formula of the maximum-log-likelihood formulation of the problem?

Thanks!

Answer 1

Just a suggestion as no other answers are sent.

You could use the normal EM with GMM (OpenCV for ex. has many wrappers for many languages) and put some points twice in the cluster you want to have "more weight". That way the EM would consider those points more important. You can remove the extra points later if it does matter.

Otherwise I think this goes quite extreme mathematics unless you have strong background in advanced statistics.

Answer 2

You can calculate a weighted log-Likelihood function; just multiply the every point with it's weight. Note that you need to use the log-Likelihood function for this.

So your problem reduces to minimizing $-\ln L = \sum_i w_i \ln f(x_i|q)$ (see the Wikipedia article for the original form).