Makeup Extraction of 3D Representation via Illumination-Aware Image Decomposition
Abstract
Facial makeup enriches the beauty of not only real humans but also virtual characters; therefore, makeup for 3D facial models is highly in demand in productions. However, painting directly on 3D faces and capturing real-world makeup are costly, and extracting makeup from 2D images often struggles with shading effects and occlusions. This paper presents the first method for extracting makeup for 3D facial models from a single makeup portrait. Our method consists of the following three steps. First, we exploit the strong prior of 3D morphable models via regression-based inverse rendering to extract coarse materials such as geometry and diffuse/specular albedos that are represented in the UV space. Second, we refine the coarse materials, which may have missing pixels due to occlusions. We apply inpainting and optimization. Finally, we extract the bare skin, makeup, and an alpha matte from the diffuse albedo. Our method offers various applications for not only 3D facial models but also 2D portrait images. The extracted makeup is well-aligned in the UV space, from which we build a large-scale makeup dataset and a parametric makeup model for 3D faces. Our disentangled materials also yield robust makeup transfer and illumination-aware makeup interpolation/removal without a reference image.
BibTeX
@article {10.1111:cgf.14762,
journal = {Computer Graphics Forum},
title = {{Makeup Extraction of 3D Representation via Illumination-Aware Image Decomposition}},
author = {Yang, Xingchao and Taketomi, Takafumi and Kanamori, Yoshihiro},
year = {2023},
publisher = {The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {10.1111/cgf.14762}
}
journal = {Computer Graphics Forum},
title = {{Makeup Extraction of 3D Representation via Illumination-Aware Image Decomposition}},
author = {Yang, Xingchao and Taketomi, Takafumi and Kanamori, Yoshihiro},
year = {2023},
publisher = {The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {10.1111/cgf.14762}
}
Collections
Related items
Showing items related by title, author, creator and subject.
-
Rational Bézier Guarding
Khanteimouri, Payam; Mandad, Manish; Campen, Marcel (The Eurographics Association and John Wiley & Sons Ltd., 2022)We present a reliable method to generate planar meshes of nonlinear rational triangular elements. The elements are guaranteed to be valid, i.e. defined by injective rational functions. The mesh is guaranteed to conform ... -
VA + Embeddings STAR: A State-of-the-Art Report on the Use of Embeddings in Visual Analytics
Huang, Zeyang; Witschard, Daniel; Kucher, Kostiantyn; Kerren, Andreas (The Eurographics Association and John Wiley & Sons Ltd., 2023)Over the past years, an increasing number of publications in information visualization, especially within the field of visual analytics, have mentioned the term ''embedding'' when describing the computational approach. ... -
Teaching Game Programming in an Upper-level Computing Course Through the Development of a C++ Framework and Middleware
Hooper, Steffan; Wünsche, Burkhard C.; Denny, Paul; Luxton-Reilly, Andrew (The Eurographics Association, 2024)The game development industry has a programming skills shortage, with industry surveys often ranking game programming as the top skill-in-demand across small, mid-sized, and large triple-A (AAA) game studios. C++ programming ...