need more complex index !

Question

I'm sorry if it's not an appropriate question for this site, and if it's necesary I'll close this question. But maybe someone could give me an ideea:

I'm trying to find a more complex index to make an hierarchy. For example:

5 votes from 6 = 83% AND

500 votes from 600 = 83%;

10 votes from 600 = 1.66%

If I make a hierarchy with the %, first two will be on the same place, but I think that 83% from 600 it's more valuable than the first one.

I could compare 5, 10, 500, but again it's not fair because the third case (10 votes) will be in front of the first case (5 votes), wich it's not fair beacuse the third case has only 1.66%

Maybe someone could give me an ideea how to give more weight for the second case but in the same time let the let the new entries have a fair chance.

Answer 1

Compare percentages and when they are equal (or very close to equal), resolve the draw by comparing vote counts.

Answer 2

This is a standard problem that calls for a Bayesian solution. You are interested in the posterior mean of the proportion of votes to observations.

The simplest approach is to model the votes as coming from a Binomial distribution and specify a conjugate Beta prior with parameters alpha and beta. This leads to a posterior mean = (votes + alpha) / (n + alpha + beta). You can see how larger alpha and beta smooth the average towards a common mean.

A better approach would be to set up a hierarchical model and estimate alpha and beta from the data. Matching moments will probably work well, although it is not fully Bayesian. This problem is isomorphic with the rats example in Gelman et al. (2003); Bolstad (2004) also has a chapter on the Binomial model. See here, here, and here.