Smooth Reconstruction and Compact Representation of Reflectance Functions for Image-based Relighting

Abstract

In this paper we present a new method to reconstruct reflectance functions for image-based relighting. A reflectance function describes how a pixel in a photograph is observed depending on the incident illumination on the depicted object. Additionally we present a compact representation of the reconstructed reflectance functions. The reflectance functions are sampled from real objects by illuminating the object from a set of directions while recording photographs. Each pixel in a photograph is a sample of the reflectance function. Next, a smooth continuous function is reconstructed, using different reconstruction techniques, from the sampled reflectance function. The presented method maintains important high frequency features such as highlights and self-shadowing and ensures visually pleasing relit images, computed with incident illumination containing high and low frequency features. The reconstructed reflectance functions and incident illumination can be expressed by a common set of basis functions, enabling a significant speed-up of the relighting process. We use a non-linear approximation of higher order wavelets to preserve the smoothness of the reconstructed signal while maintaining good relit image quality. Our method improves on visual quality in comparison with previous image-based relighting methods, especially when animated incident illumination is used.