Minimal Convolutional Neural Networks for Temporal Anti Aliasing

Abstract

Existing deep learning methods for performing temporal anti aliasing (TAA) in rendering are either closed source or rely on upsampling networks with a large operation count which are expensive to evaluate. We propose a simple deep learning architecture for TAA combining only a few common primitives, easy to assemble and to change for application needs. We use a fully-convolutional neural network architecture with recurrent temporal feedback, motion vectors and depth values as input and show that a simple network can produce satisfactory results. Our architecture template, for which we provide code, introduces a method that adapts to different temporal subpixel offsets for accumulation without increasing the operation count. To this end, convolutional layers cycle through a set of different weights per temporal subpixel offset while their operations remain fixed. We analyze the effect of this method on image quality and present different tradeoffs for adapting the architecture. We show that our simple network performs remarkably better than variance clipping TAA, eliminating both flickering and ghosting without performing upsampling.