Improving Depth Estimation Using Superpixels

Abstract

This work is focused on assigning a depth label to each pixel in the image. We consider off-the-shelf algorithms that provide depth information from multiple views or depth information directly obtained from RGB-d sensors. Both of them are common scenarios of a well studied problem where many times we get incomplete depth information. Then, user interaction becomes necessary to finish, improve or correct the solution for certain applications where accurate and dense depth information for all pixels in the image is needed. This work presents our approach to improve the depth assigned to each pixel in an automated manner. Our proposed pipeline combines state-of-the art methods for image superpixel segmentation and energy minimization. Superpixel segmentation reduces complexity and provides more robustness to the labeling decisions. We study how to propagate the depth information to incomplete or inconsistent regions of the image using a Markov Random Field (MRF) energy minimization framework. We propose and evaluate an energy function and validate it together with the designed pipeline. We present a quantitative evaluation of our approach with different variations to show the improvements we can obtain. This is done using a publicly available stereo dataset that provides ground truth information. We show additional qualitatively results, with other tests cases and scenarios using different input depth information, where we also obtain significant improvements on the depth estimation compared to the initial one.