Seeing Through Sounds: Mapping Auditory Dimensions to Data and Charts for People with Visual Impairments

Abstract

Sonification can be an effective medium for people with visual impairments to understand data in visualizations. However, there are no universal design principles that apply to various charts that encode different data types. Towards generalizable principles, we conducted an exploratory experiment to assess how different auditory channels (e.g., pitch, volume) impact the data and visualization perception among people with visual impairments. In our experiment, participants evaluated the intuitiveness and accuracy of the mapping of auditory channels on different data and chart types. We found that participants rated pitch to be the most intuitive, while the number of tappings and the length of sounds yielded the most accurate perception in decoding data. We study how audio channels can intuitively represent different charts and demonstrate that data-level perception might not directly transfer to chart-level perception as participants reflect on visual aspects of the charts while listening to audio. We conclude by how future experiments can be designed to establish a robust ranking for creating audio charts.