An Inverted Pyramid Acceleration Structure Guiding Foveated Sphere Tracing for Implicit Surfaces in VR

Abstract

In this paper, we propose a novel rendering pipeline for sphere tracing signed distance functions (SDFs) that significantly improves sphere tracing performance. Previous methods simply focus on over-relaxing the step size by a fixed amount and thus reducing the total step count of the ray based on the error of the previous step at the full rendering resolution. Unlike those, our system reconstructs the final image in a multi-scale inverted pyramid fashion that provides progressively finer approximations of a surface's distance from the camera origin. We initiate sphere tracing at a very low resolution approximation of the scene which provides an initial estimate of the closest surface to a group of rays to be sphere traced. We shoot and trace those rays from that approximated distance instead of shooting them from the camera origin, providing a massive head-start for the rays to leap ahead in the 3D scene, successively generating the following level until the full resolution is reached. This significantly reduces the total step count. Moving up in the pyramid in higher and higher resolutions we repeat this process to further eliminate sphere tracing steps. The multiple resolution levels of the pyramid ascertain that we avoid jumps of the ray in the 3D scene that would potentially generate artefacts, especially around scene edges that might be missed when rendering at lower resolutions. This approach allows for a much more efficient use of computational resources and results in a significant boost in performance (more than 20x speed-up in some cases). Integrating a foveated rendering algorithm within the inverted pyramid pipeline further accelerates performance enabling 16x super-sample anti-aliasing of implicit surfaces in a VR headset. Our experiments demonstrate that our image manipulation remains imperceptible. Our benchmark evaluation indicated a significant boost in sphere tracing performance with or without foveated rendering applied. This enables efficiently rendering SDFs in VR headsets, often otherwise impossible due to limited performance.