Effectiveness evaluation is of paramount importance in Information Retrieval (IR). The effectiveness metric being used is a fundamental parameter in any evaluation, and metric choice is neither a simple task, nor it is without consequences: an inadequate metric might mean to waste research efforts improving systems toward a wrong target. The problem is exacerbated by the large number of metrics existing, more than one hundred when counting the system-oriented metrics only. In this talk I will introduce some of the most common IR metrics and propose an axiomatic approach to metrics. I will then present a general framework based on Measurement theory that can be used to express axioms, thus making explicit the desirable metric properties, and will hopefully allow to better understand them.
Lecturer: Professor Stefano Mizzaro
