arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Computer Science · Computer Vision and Pattern Recognition

A Personalized Talking/Moving Head Presentation Using Image-based Transformations

Ali Arya, Babak Hamidzadeh

See on the map

Abstract

This paper addresses the problem of multimedia presentation of a moving and talking head. Existing approaches either use a 3D geometrical representation or multiple 2D views to model and reconstruct faces. To achieve realistic appearance and personalization, and avoid complex models, we propose two approaches. First, we show how pure image-based view morphing can be used to create new moving and talking faces based on existing footage. These views form a video stream which in turn will be synchronized with the output of a personalized Text-To-Speech system. Then, to improve this approach, we use facial feature detection to fine tune the correlation-based optical flow computation used for view morphing, and also create talking views of a new person based on one non-talking view in any position and talking views of a reference person. 1.

More like this