Choosing the Right Sample? Experiences of Selecting Participants for Visualization Evaluation

Abstract

Conducting and reporting evaluation studies has become more and more popular over the last few years in the information visualization community. A big challenge is to describe such studies in a way such that the investigations are repeatable and comparable with other studies. This not only includes the description of methodology, tasks, and procedure of the study but also information about the participants - including the reasons for their selection - to make the work reproducible and to assess its validity. In this paper we give a short overview about our research that we conducted in the past to show in which context and situations which types of test persons (e.g., students or experts) were considered.