Template-Based 3D Non-Rigid Shape Estimation from Monocular Image Sequences

Abstract

This paper addresses the problem of reconstructing non-rigid 3D geometries from temporal image sequences captured with only a single camera under full perspective projection. Without the knowledge of a shape deformation model, this task is severly under-constrained, because multiple shape configurations can produce the same image projections. The challenge remains even if a template 3D model of the static, un-deformed state is available, because the depth along the line of sight is unkown. Often, this is handled by assuming an orthographic camera model. In contrast, we address a full perspective camera model. Also, our reconstruction is not limited to the model parts that are visible in the current image, but deformation is estimated for the entire template across the temporal sequence. In a first step, we compute a template of the geometry in un-deformed pose, assuming that the object was captured while being static. Next, the object starts to deform while being captured by a single camera, and the non-rigid shape is reconstructed sequentially by estimating the camera position and the deformations with respect to the template model. Our objective minimization function combines image data and temporal consistency information, and constrains the deformation space by a rotation-invariant volumetric graph Laplacian and as-rigid-as-possible constraints defined on the tesselation of the template model. The method is evaluated on synthetic and real data, including different object classes, thereby concentrating on the class of articulated deformations.