A Hardware-Friendly Bilateral Solver Accelerator for Real-Time Virtual Reality Video

Abstract

Rendering 3D-360° VR video from a camera rig is computationintensive and typically performed o ine. In this paper, we target the most time-consuming step of the VR video creation process, high-quality ow estimation with the bilateral solver.We propose a new algorithm, the hardware-friendly bilateral solver, that enables faster runtimes than existing algorithms of similar quality. Our algorithm is easily parallelized, achieving a 4 speedup on CPU and 32 speedup on GPU over a baseline CPU implementation. We also design an FPGA-based hardware accelerator that utilizes reduced-precision computation and the parallelism inherent in our algorithm to achieve further speedups over our CPU and GPU implementations while consuming an order of magnitude less power. e FPGA design's power e ciency enables practical real-time VR video processing at the camera rig or in the cloud.