Extending Document Exploration with Image Retrieval: Concept and First Results

Abstract

Information retrieval provides to date effective methods to search for documents relevant to user queries, and to support exploration of clusters of similar documents. Typically, the retrieval relies on text-based queries and similarity functions. However, in many cases also visual content is important in documents, for example, in the visualization field. There, researchers may want to search for papers based on similar example visualizations, which is difficult by relying on keyword search alone. We present a concept to automatically label visualization types in research papers and search for similar images, relying on state of the art image descriptors. We created a prototype that allows to search for papers showing images similar to a query image. Preliminary results of applying it on a corpus of VAST papers indicate the chosen descriptors can retrieve papers with similar images. Our approach for image-based search can complement text-based search and in perspective, support document corpus exploration based on clustering contained images. In future work, we want to explore if image-based search can also support the formation of taxonomies of a corpus or research papers, based on image similarity.