What is the 11pt average precision metric?

Question

Hello

I have two questions

1- what is the "11pt average precision metric"?

2- and how to use it in information retrieval?

Thanks

Answer 1

Sometimes I wonder why folks ask such questions where when google can answer them better than many humans. Then google leads me to this Wikipedia article and I drown is a sea of formulae and densly defined terms.

As far as I can see 11 point average precision is an evaluation of how well an automated system has categorised a set of documents. So that's relevent to information retrieval because it gives you a metric for evaluating the usefulness of (for example) a search engine.

So, if you can understand that article, there's your answer. Otherwise here's hoping that you and I get the benefit of someone explaining it to us :-)

Good luck.

Answer 2

You should consult the book, "Introduction to Modern Information Retrieval" by Salton & McGill. The following online reference has a decent summary of the metric, it's use and an algorithm:

• http://datamin.ubbcluj.ro/wiki/index.php/Evaluation%5Fmethods%5Fin%5Ftext%5Fcategorization#11-point%5Faverage%5Fprecision

Answer 3

Here is an online book, Introduction to Information Retrieval
and there is a reference to 11-point interpolated average precision use, evaluation of ranked retrieval results.

Answer 4

This comes mainly from the book "Introduction to Information Retrieval" by Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze.

In ranked retrieval, you have precision and recall. Precision is how accurate the system is in the documents it returns (1.0 means it only returned relevant documents). Recall is what percentage of the relevant documents the system found (1.0 means it found them all). So you can adjust the recall of the system by returning more and more documents, but there is a tradeoff with precision, because you are likely to admit more error.

Interpolated precision is where you pick a recall level r and for all recall levels r' >= r, it is the best precision you can achieve.

Then in 11-pt interpolated average precision, you are looking at 11 recall levels (0.0, 0.1, 0.2, ..., 1.0) and finding the interpolated precision at each point. You average these scores across all of the different queries or information needs (as the Manning & Schuetze book put it) to find your system's score.

The typical usage in information retrieval is for evaluation. It helps you compare one system to another, but where it really helps is in letting you compare how your system is changing as you tweak parameters. Does changing this setting improve or damage my 11-pt interp avg prec? The benefit is, it measures across all information needs in your collection, so it helps to give you a "big picture" view. But, be careful about relying too heavily on these metrics since they are only as good as the evaluation set you are using.