Volumetric Hand Reconstructions for Non-verbal Communication in Shared Virtual Environments

Abstract

Enhancing desk-based computer environments with virtual reality technology requires natural interaction support, in particular hand tracking and visualization. In shared virtual environments hand tracking does not only support interaction with the virtual environment, but also provides a channel for non-verbal communication. This paper presents a tabletop user interface and its integration into a three-dimensional teleconferencing system. Dense volume reconstructions of hands based on a probabilistic Shape from Silhouette algorithm are applied for tracking and visualization. The interface integration employs the derived probabilistic hand reconstructions to interactively generate mesh models characterized as isosurfaces of probabilistic volume data. It will be shown, that meshes resulting from probabilistic volume reconstructions thereby exhibit superior detail in comparison to mesh models generated from non-probabilistic comparative approaches which represent the current state of the art.