Recolorable Posterization of Volumetric Radiance Fields Using Visibility-Weighted Palette Extraction

Abstract

Volumetric radiance fields have recently gained significant attention as promising representations of photorealistic scene reconstruction. However, the non-photorealistic rendering of such a representation has barely been explored. In this study, we investigate the artistic posterization of the volumetric radiance fields. We extend the recent palette-based image-editing framework, which naturally introduces intuitive color manipulation of the posterized results, into the radiance field. Our major challenge is applying stylization effects coherently across different views. Based on the observation that computing a palette frame-by-frame can produce flickering, we propose pre-computing a single palette from the volumetric radiance field covering its entire visible color. We present a method based on volumetric visibility to sample visible colors from the radiance field while avoiding occluded and noisy regions. We demonstrate our workflow by applying it to pre-trained volumetric radiance fields with various stylization effects. We also show that our approach can produce more coherent and robust stylization effects than baseline methods that compute a palette on each rendered view.