Simple Techniques for a Novel Human Body Pose Optimisation Using Differentiable Inverse Rendering

Abstract

Human body 3D reconstruction has a wide range of applications including 3D-printing, art, games, and even technical sport analysis. This is a challenging problem due to 2D ambiguity, diversity of human poses, and costs in obtaining multiple views. We propose a novel optimisation scheme that bypasses the prior bias bottleneck and the 2D-pose annotation bottleneck that we identify in single-view reconstruction, and move towards low-resource photo-realistic 3D reconstruction that directly and fully utilises the target image. Our scheme combines domain-specific method SMPLify-X and domain-agnostic inverse rendering method redner, with two simple yet powerful techniques. We demonstrate that our techniques can 1) improve the accuracy of the reconstructed body both qualitatively and quantitatively for challenging inputs, and 2) control optimisation to a selected part only. Our ideas promise extension to more difficult problems and domains even beyond human body reconstruction.