Video-Realistic Inner Mouth Reanimation

Abstract

We propose a novel post-effect method that can make an existing speech animation video-realistic by generating an inner mouth appearance that is tailored to the speaker. The automatic generation of photorealistic inner mouth appearances requires only simple inputs and small databases and is not restricted by video lighting. The approach is also applicable for creature speech animation with human voices. Our system uses two key algorithms; one of the algorithms models the inner mouth appearances based on physical assumptions, and the other algorithm synthesizes the inner mouth images with care taken to maintain time continuity and luminance gradients.