Simulation of Time-of-Flight Sensors using Global Illumination

Abstract

Time-of-Flight (ToF) cameras use specialized sensors and modulated infrared light to simultaneously obtain depth, amplitude and intensity images. Depth images from such cameras suffer from various errors which exhibit a more complex behavior than traditional intensity images. Of these errors, the phenomenon of multi-reflection or multipath interference poses the biggest challenge to researchers. It is caused by indirect light paths between camera and light source and is therefore dependent on scene geometry. While simulated data can be used for ground truth evaluation and whitebox testing, current simulators do not model multipath effects. The method we present is capable of simulating all scene-dependant effects by taking global illumination into consideration. This is accomplished by modifying a bidirectional path tracing algorithm such that it takes the time-dependent propagation of modulated light in a scene into consideration. Furthermore, by combination of the proposed method with a previous hardware simulator we are capable of reproducing all effects in ToF cameras. The system was validated both on test targets with known real Time of Flight camera responses as well as qualitatively on a more complex room scene. The simulator as well as the source code is available at http://hci.iwr.uni-heidelberg.de/Benchmarks/.