Using Topic Ontologies and Semantic Similarity Data to Evaluate Topical Search

MAGUITMAN ANA GABRIELA; CECCHINI ROCIO LUJÁN; LORENZETTI CARLOS MARTÍN; MENCZER FILLIPO

Asunción

Conferencia; Conferencia Latinoamericana de Informática; 2010

Centro Latinoamericano de Estudios en Informática

Improvement in information retrieval systems is largely dependent on the ability to evaluate them. In order to assess the effectiveness of a retrieval system, test collections are needed. In traditional approaches users or hired evaluators provide manual assessments of relevance. However this does not scale with the complexity and heterogeneity of available digital information. This paper proposes to use topic ontologies and semantic similarity data to alleviate the efforts needed by human assessors to evaluate the rapidly expanding set of competing information retrieval methods. After providing experimental evidence supporting the validity of our approach we illustrate its application with an example in which the proposed evaluation procedure is used to assess the effectiveness of topical retrieval systems.