RETRIEVAL 3D: An On-line Content-Based Retrieval Performance Evaluation Tool

Abstract

Performance benchmarking is an absolute necessity when attempting to objectively quantify the performance of content-based retrieval methods. For many years now, a number of plot-based and scalar-based measures in combination with benchmark datasets have already been used in order to provide objective results. In this work, we present the first version of an integrated on-line content-based retrieval evaluation tool, named RETRIEVAL 3D, which can be used in order to quantify the performance of a retrieval method. The current version of the system offers a set of popular performance measures that can be accessed through a dynamic visualisation environment. The user is able to upload retrieval results using different input data structures (e.g. binary ranked lists, floating point ranked lists, dissimilarity matrices and groundtruth data) that are already encountered in the literature including the SHREC competition series. Moreover, the system is able to provide evaluation mechanisms for known within the retrieval research community benchmark datasets. It offers performance measures parameterisation that enables the user to determine specific aspects of the evaluated retrieval method. Performance reports archiving and downloading are some of the system's user-oriented functionalities.