Visual Comparison of Ranked Result Cumulated Gains

Abstract

Ranking is fundamental in Information Retrieval (IR) and several measures have been developed over the years to assess the quality of a ranked result list, such as those based on the idea of computing the cumulative gain up to a given ranked position and taking into account multiple relevance levels. These measures allow for comparing the performances of different Information Retrieval System (IRS), giving credit to their ability to retrieve highly relevant documents and to rank them topmost in the result list. However, while this approach is able to assess the differences among two or more retrieval systems, it does not allow to easily understand and inspect the reasons of good or bad performances. To this end, this paper presents a Visual Analytics (VA) environment that allows for visually exploring the ranked retrieval results, pointing out the search failures and providing useful insights for improving the underlying IRS ranking algorithm.