Image Based Interactive Rendering with View Dependent Geometry

Abstract

In this paper we present a novel approach for interactive rendering of virtual views from real image sequences.Combining the concepts of light fields, depth-compensated image warping and view dependent texture mapping,this plenoptic modeling approach can handle large and complex scenes. A portable, handheld multi-camera systemhas been developed that allows to record multiple image streams by simply walking around the scene. Theseimage streams are automatically calibrated and depth maps for all views are generated as input to the renderingstage. For rendering a view dependent warping surface is constructed on the fly and depth-compensated imageinterpolation is applied with view-dependent texture mapping. Rendering quality is scalable to allow fast previewand to achieve high-end quality with the same approach. The system can handle large and geometrically complexscenes with hundreds of real images at interactive rates.Categories and Subject Descriptors (according to ACM CCS): I.3.3 [Computer Graphics]: Viewing algorithms, I.4.1[Image Processing and Computer Vision]: Digitization and Image Capture, I.4.8 [Image Processing and ComputerVision]: Scene Analysis