Depth Enhanced Panoramas

Abstract

Depth enhanced panoramas are a practical solution to the difficult problem of inside-looking-out modeling. Depth enhanced panoramas extend color panoramas to support viewpoint translation, while retaining their speed, convenience, and low cost. Depth enhanced panoramas are built incrementally from same-center-of-projection dense-color and sparse-depth frames that are acquired, registered, and merged at the rate of 5 frames per second. The evolving depth enhanced panorama is rendered continually to provide immediate operator feedback. The viewpoint translation range is increased by combining multiple depth enhanced panoramas in real time. Depth enhanced panoramas are combined using either a splat-based, disconnected representation, or, at pixel level, using a GPU fragment program. Depth enhanced panoramas are built in minutes with $3,000 hardware and support photorealistic rendering of complex room-sized environments.