Variational Separation of Light Field Layers

Abstract

Images of scenes which contain reflective or transparent surfaces are composed of different layers which are observed at different depths. Analyzing such a scene requires separating the image into its individual layers, which remains a challenging and important problem. While the problem is very much ill-posed when only a single image is considered, recent work has shown that depth estimation for two layers becomes quite tractable when one instead captures a 4D light field of the scene. In this paper, we propose a novel variational approach to layer separation which is based on these ideas. We formulate a linear generative model to reconstruct the light field from disparity and luminance information for the individual layers on the center view. Comparing the model with the observerd data yields a convex variational problem for layer reconstruction, which can be solved to global optimality with a primal-dual scheme. Layer disparity is estimated in a first step, for which we improve upon a model based on second order structure tensors on the epipolar plane images. In contrast to previous work, the resulting approach is robust enough to be able to deal with light fields from the Lytro Illum camera, for which we obtain a compelling separation of the reflectance layer in real-world scenes.