arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Computer Science · Computer Vision and Pattern Recognition

Abstract Real-Time Speech-Driven Face Animation

Pengyu Hong, Zhen Wen, Tom Huang

See on the map

Abstract

This chapter presents our research on real-time speech-driven face animation. First, a visual representation, called Motion Unit (MU), for facial deformation is learned from a set of labeled face deformation data. A facial deformation can be approximated by a lin-ear combination of MUs weighted by the corresponding MU parameters (MUPs), which are used as the visual features of facial deformations. MUs explore the correlation among those facial feature points used by the MPEG-4 face animation (FA) to describe facial deformations. MU-based FA is compatible with MPEG-4 FA. We then collect a set of audio-visual (AV) training database and use the training database to train a real-time audio-to-visual mapping (AVM). 1.

More like this