Pose Space Image Based Rendering

Abstract

This paper introduces a new image-based rendering approach for articulated objects with complex pose-dependent appearance, such as clothes. Our approach combines body-pose-dependent appearance and geometry to synthesize images of new poses from a database of examples. A geometric model allows animation and view interpolation, while small details as well as complex shading and reflection properties are modeled by pose-dependent appearance examples in a database. Correspondences between the images are represented as mesh-based warps, both in the spatial and intensity domain. For rendering, these warps are interpolated in pose space, i.e. the space of body poses, using scattered data interpolation methods. Warp estimation as well as geometry reconstruction is performed in an offline procedure, thus shifting computational complexity to an a-priori training phase.