Guidelines and Recommendations for the Evaluation of New Visualization Techniques by Means of Experimental Studies

Abstract

This paper addresses important issues in the evaluation of new visualization techniques. It describes the principle of quantitative research in general and presents the idea of experimental studies. The goal of experimental studies is to provide the base for guidelines, which allow testing of hypotheses that newly-developed visualization solutions are better than older ones. Moreover, the paper provides guidelines for successful planning of experimental studies in terms of independent and dependent variables, participants, tasks, data collection and statistical evaluation of collected data. It describes how the results should be interpreted and reported in publications. Finally, the paper points out useful literature and thus contributes to a better understanding of how to evaluate new visualization techniques.