dc.contributor.author | Li, Gengyan | en_US |
dc.contributor.author | Sarkar, Kripasindhu | en_US |
dc.contributor.author | Meka, Abhimitra | en_US |
dc.contributor.author | Buehler, Marcel | en_US |
dc.contributor.author | Mueller, Franziska | en_US |
dc.contributor.author | Gotardo, Paulo | en_US |
dc.contributor.author | Hilliges, Otmar | en_US |
dc.contributor.author | Beeler, Thabo | en_US |
dc.contributor.editor | Bermano, Amit H. | en_US |
dc.contributor.editor | Kalogerakis, Evangelos | en_US |
dc.date.accessioned | 2024-04-16T14:42:13Z | |
dc.date.available | 2024-04-16T14:42:13Z | |
dc.date.issued | 2024 | |
dc.identifier.issn | 1467-8659 | |
dc.identifier.uri | https://doi.org/10.1111/cgf.15041 | |
dc.identifier.uri | https://diglib.eg.org:443/handle/10.1111/cgf15041 | |
dc.description.abstract | Eye gaze and expressions are crucial non-verbal signals in face-to-face communication. Visual effects and telepresence demand significant improvements in personalized tracking, animation, and synthesis of the eye region to achieve true immersion. Morphable face models, in combination with coordinate-based neural volumetric representations, show promise in solving the difficult problem of reconstructing intricate geometry (eyelashes) and synthesizing photorealistic appearance variations (wrinkles and specularities) of eye performances. We propose a novel hybrid representation - ShellNeRF - that builds a discretized volume around a 3DMM face mesh using concentric surfaces to model the deformable 'periocular' region. We define a canonical space using the UV layout of the shells that constrains the space of dense correspondence search. Combined with an explicit eyeball mesh for modeling corneal light-transport, our model allows for animatable photorealistic 3D synthesis of the whole eye region. Using multi-view video input, we demonstrate significant improvements over state-of-the-art in expression re-enactment and transfer for high-resolution close-up views of the eye region. | en_US |
dc.publisher | The Eurographics Association and John Wiley & Sons Ltd. | en_US |
dc.rights | Attribution 4.0 International License | |
dc.rights.uri | https://creativecommons.org/licenses/by/4.0/ | |
dc.subject | CCS Concepts: Computing methodologies -> Motion capture; Physical simulation; Image-based rendering; Mixed / augmented reality; Volumetric models; Parametric curve and surface models; Appearance and texture representations; Shape representations; 3D imaging; Reconstruction | |
dc.subject | Computing methodologies | |
dc.subject | Motion capture | |
dc.subject | Physical simulation | |
dc.subject | Image | |
dc.subject | based rendering | |
dc.subject | Mixed / augmented reality | |
dc.subject | Volumetric models | |
dc.subject | Parametric curve and surface models | |
dc.subject | Appearance and texture representations | |
dc.subject | Shape representations | |
dc.subject | 3D imaging | |
dc.subject | Reconstruction | |
dc.title | ShellNeRF: Learning a Controllable High-resolution Model of the Eye and Periocular Region | en_US |
dc.description.seriesinformation | Computer Graphics Forum | |
dc.description.sectionheaders | Face Modeling and Reconstruction | |
dc.description.volume | 43 | |
dc.description.number | 2 | |
dc.identifier.doi | 10.1111/cgf.15041 | |
dc.identifier.pages | 15 pages | |