Towards A 3D Virtual Studio for Human Appearance Capture

Abstract

This paper introduces the concept of a ''3D Virtual Studio'' for human appearance capture, akin to the motion capture studio for human motion tracking. Ultimately the 3D Virtual Studio should enable video-realistic reconstruction of a moving person from any viewpoint. A mesh-based stereo technique is presented to reconstruct a moving person from multiple camera views. This technique optimises a surface mesh to match stereo and silhouette data in a constrained coarse-to-fine framework, recovering sub-pixel image correspondence in the presence of inexact camera calibration. We compare this approach for scene reconstruction to conventional shape from silhouette and multiple view stereo. We then demonstrate view-dependent rendering and show improved resolution with the recovered image correspondence. We then demonstrate how this approach can be used to capture the dynamic shape and appearance of a computer graphics model of a person.