Proxy Clouds for RGB-D Stream Processing: A Preview

Abstract

Modern consumer depth cameras are widely used for 3D capture in indoor environments, for applications such as modeling, robotics or gaming. Nevertheless, their use is limited by their low resolution, with frames often corrupted with noise, missing data and temporal inconsistencies. In order to cope with all these issues, we present Proxy Clouds, a multiplanar superstructure for real-time processing of RGB-D data. By generating a single set of planar proxies from raw RGB-D data and updating it through time, several processing primitives can be applied to improve the quality of the RGB-D stream or lighten further operations. We illustrate the use of Proxy Clouds on several applications, including noise and temporal flickering removal, hole filling, resampling, color processing and compression. We present experiments performed with our framework in indoor scenes of different natures captured with a consumer depth sensor.