Material Editing in Complex Scenes by Surface Light Field Manipulation and Reflectance Optimization

Abstract

This work addresses the challenge of intuitive appearance editing in scenes with complex geometric layout and complex, spatially-varying indirect lighting. In contrast to previous work, that aimed to edit surface reflectance, our system allows a user to freely manipulate the surface light field. It then finds the best surface reflectance that ''explains'' the surface light field manipulation. Instead of classic L2 fitting of reflectance to a combination of incoming and exitant illumination, our system infers a sparse L0 change of shading parameters instead. Consequently, our system does not require ''diffuse'' or ''glossiness'' brushes or any such understanding of the underlying reflectance parametrization. Instead, it infers reflectance changes from scribbles made by a single simple color brush tool alone: Drawing a highlight will increase Phong specular; blurring a mirror reflection will decrease glossiness; etc. A sparse-solver framework operating on a novel point-based, pre-convolved lighting representation in combination with screen-space edit upsampling allows to perform editing interactively on a GPU.