# question about probability ranking principle

I am studying IR and

I am not clearly understand "probability ranking principle" ( I tried to google the definition, but i couldn't find clear answer.)

I am assuming that it's system which response to each request is a ranking of the documents in the collection in order of decreasing probability.

but, I am not clearly understand concept of probability ranking principle

does anyone can explain it ?

thanks

• What is IR? Also, can you gives us at least one of the definitions of "probability ranking system"? – Tyson Williams Sep 22 '12 at 12:26
• IR is information retrieval. – rrenaud Sep 22 '12 at 15:26

## 2 Answers

This is a clear definition of the probability ranking principle.

If a reference retrieval system's response to each request is a ranking of the documents in the collection in order of decreasing probability of relevance to the user who submitted the request, where the probabilities are estimated as accurately as possible on the basis of whatever data have been made available to the system for this purpose, the overall effectiveness of the system to its user will be the best that is obtainable on the basis of those data.

As an IR professional (I work on Google Local Search), the principle is definitely wrong in the case that you assume the searcher doesn't examine every result and is just looking for one to satisfy their request. Imagine you have 10 results that are excellent independently, but are near duplicates of each other. Then the probability that the user finds something relevant barely increases as you spend results on the duplicate documents. If you would have shown a diverse set of results with just one of the excellent, non-duplicate results, and others that satisfy the a secondary interpretation of the query, you'd do better to satisfy the user.

On the other hand, if your evaluation metrics for a set of results are just a sum of independent, result level metrics, then the principle is certainly correct. You want to maximize the relevance of each result, and you can do that by picking the best ones in a vacuum.

The probability ranking principle asserts that relevance has a probabilistic interpretation. According to this principle documents are ranked by a probability p(Rel|d, q), where Rel denotes the event of a document d being relevant to a query q. Robertson called this principle the probability ranking principle.

• It would be better to use latex for math symbols and provide some reference. – Yixin Cao Dec 4 '13 at 23:38