Visual Analytics for Information Retrieval Evaluation Campaigns

Abstract

Information Retrieval (IR) has been deeply rooted in experimentation since its inception, allowing researchers and developers to understand the behavior and interactions within increasingly complex IR systems, such as web search engines, which have to address ever increasing user needs and support challenging tasks. This paper focuses on the innovative Visual Analytics (VA) approach realized by the Participative Research labOratory for Multimedia and Multilingual Information Systems Evaluation (PROMISE) environment, which simplifies and makes more effective the experimental evaluation process by allowing a formal and structured way to explore the complex data set of measures produced along an evaluation campaign. The system uses the result produced within the Conference and Labs of the Evaluation Forum (CLEF) [Cle].