Support for inferring user abilities for multimodal applications

Abstract

Multimodal interaction offers users the possibility to interact with computational systems using natural interaction modes. However, different users interact differently, integrating modalities in different patterns. By knowing how users exploit different modalities, we can try to build adaptive systems that will increase the effectiveness of multimodal interaction. For that we need to characterize user abilities. However, this is a time and effort consuming process, particularly in the early stages of design. We propose an approach that can be used in the early stages of the design process, by resorting to a Wizard of Oz based prototype of a multimodal system.