An Ontological Framework for Supporting the Design and Evaluation of Visual Analytics Systems

Abstract

Designing, evaluating, and improving visual analytics (VA) systems is a primary area of activities in our discipline. In this paper, we present an ontological framework for recording and categorizing technical shortcomings to be addressed in a VA workflow, reasoning about the causes of such problems, identifying technical solutions, and anticipating secondary effects of the solutions. The methodology is built on the theoretical premise that designing a VA workflow is an optimization of the costbenefit ratio of the processes in the workflow. It makes uses three fundamental measures to group and connect ''symptoms'', ''causes'', ''remedies'', and ''side-effects'', and guide the search for potential solutions to the problems. In terms of requirement analysis and system design, the proposed methodology can enable system designers to explore the decision space in a structured manner. In terms of evaluation, the proposed methodology is time-efficient and complementary to various forms of empirical studies, such as user surveys, controlled experiments, observational studies, focus group discussions, and so on. In general, it reduces the amount of trial-and-error in the lifecycle of VA system development.