A Robust Multi-View System for High-Fidelity Human Body Shape Reconstruction

Abstract

This paper proposes a passive multi-view system for human body shape reconstruction, namely RHF-Human, to overcome several challenges including accurate calibration and stereo matching in self-occluded and low-texture skin regions. The reconstruction process includes four steps: capture, multi-view camera calibration, dense reconstruction, and meshing. The capture system, which consists of 90 digital single-lens reflex cameras, is single-shot to avoid nonrigid deformation of the human body. Two technical contributions are made: (1) a two-step robust multi-view calibration approach that improves calibration accuracy and saves calibration time for each new human body acquired and (2) an accurate PatchMatch multi-view stereo method for dense reconstruction to perform correct matching in self-occluded and low-texture skin regions and to reduce the noise caused by body hair. Experiments on models of various genders, poses, and skin with different amounts of body hair show the robustness of the proposed system. A high-fidelity human body shape dataset with 227 models is constructed, and the average accuracy is within 1.5 mm. The system provides a new scheme for the accurate reconstruction of nonrigid human models based on passive vision and has good potential in fashion design and health care.