arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Dmitri Bitouk

8 papers · 2001–2008

2001: 12002: 02003: 12004: 02005: 02006: 02007: 22008: 4
2001peak 4 in 20082008

Frequent co-authors

Papers

showing 1–8 of 8 RSS

Face swapping

Dmitri Bitouk, Neeraj Kumar, Samreen Dhillon +2 more

Computer Vision and Pattern Recognition · ACM Transactions on Graphics · · cited 338× · DOI ↗ · more like this ·

In this paper, we present a complete system for automatic face replacement in images. Our system uses a large library of face images created automatically by downloading images from the internet, extracting faces using face detection software, and aligning each extracted face to a common coordinate system. This library is constructed off-line, once, and can be efficiently accessed during face replacement. Our replacement algorithm has three main stages. First, given an input image, we detect all faces that are present, align them to the coordinate system used by our face library, and select candidate face images from our face library that are similar to the input face in appearance and pose. Second, we adjust the pose, lighting, and color of the candidate face images to match the appearance of those in the input image, and seamlessly blend in the results. Third, we rank the blended…

Creating a Speech Enabled Avatar from a Single Photograph

Dmitri Bitouk, Shree K. Nayar

Computer Vision and Pattern Recognition · · cited 8× · DOI ↗ · more like this ·

This paper presents a complete framework for creating a speech-enabled avatar from a single image of a person. Our approach uses a generic facial motion model which represents deformations of a prototype face during speech. We have developed an HMM-based facial animation algorithm which takes into account both lexical stress and coarticulation. This algorithm produces realistic animations of the prototype facial surface from either text or speech. The generic facial motion model can be transformed to a novel face geometry using a set of corresponding points between the prototype face surface and the novel face. Given a face photograph, a small number of manually selected features in the photograph are used to deform the prototype face surface. The deformed surface is then used to animate the face in the photograph. We show several examples of avatars that are driven by text and speech…

Head-pose and illumination invariant three-dimensional audio-visual speech recognition

Michael I. Miller, Dmitri Bitouk

Signal Processing · · cited 2× · OpenAlex ↗ · more like this ·

Speech perception is bimodal, employing not only the acoustic signal, but also visual cues. Audio-visual speech recognition aims to improve the performance of conventional automated speech recognition by incorporating visual information. Due to a fundamentally limited two-dimensional representation employed, current approaches for visual feature extraction lack invariance to speaker's pose and illumination in the environment. The research presented in this thesis aims to develop three-dimensional methods for visual feature extraction that alleviate the above-mentioned limitation. Following the concepts of Grenander's General Pattern Theory, the prior knowledge of speaker's face is described by a prototype, which consists of a 3-D surface and a texture. The variability in observed video images of a speaker associated with pose, articulatory facial motion, and illumination is represented…

Speech Enabled Avatar from a Single Photograph

Dmitri Bitouk, Shree K. Nayar

Computer Vision and Pattern Recognition · Columbia Academic Commons (Columbia University) · · cited 1× · DOI ↗ · more like this ·

This paper presents a complete framework for creating speech-enabled 2D and 3D avatars from a single image of a person. Our approach uses a generic facial motion model which represents deformations of the prototype face during speech. We have developed an HMM-based facial animation algorithm which takes into account both lexical stress and coarticulation. This algorithm produces realistic animations of the prototype facial surface from either text or speech. The generic facial motion model is transformed to a novel face geometry using a set of corresponding points between the generic mesh and the novel face. In the case of a 2D avatar, a single photograph of the person is used as input. We manually select a small number of features on the photograph and these are used to deform the prototype surface. The deformed surface is then used to animate the photograph. In the case of a 3D…

Fisher information in transported generator clutter models

Dmitri Bitouk, Ulf Grenander, Michael I. Miller +1 more

Artificial Intelligence · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · · cited 1× · DOI ↗ · more like this ·

Our work focuses on automated recognition of target obscured by clutter. Considering clutter as a filtered marked point Poisson process, a MMSE pose estimator of rigid objects in obscuring clutter is introduced. We also build asymptotic approximations for Bayesian posterior distributions based on the Fisher information. We relate the Fisher information to the Hilber-Schmidt pose estimator, the expected error of which is shown to be a lower bound on the error incurred by any other pose estimator.

Send correspondence to:

Anqi Qiu, Dmitri Bitouk, Michael I. Miller +2 more

Applied Mathematics · · cited 0× · OpenAlex ↗ · more like this ·

Functional and structural maps, such as a curvature, cortical thickness, and func-tional MRI maps, indexed over the local coordinates of the cortical manifold play an important role in neuropsychiatric studies. Due to the highly convoluted nature of the cerebral cortex and image quality, these functions are generally uninterpretable without proper methods of association and smoothness onto the local coordinate system. We propose an approach for representing functions by orthonormal bases of the Laplace-Beltrami operator with Neumann boundary conditions. The explicit discrete repre-sentation of the Laplace-Beltrami operator is derived using the finite element method calculated directly on the manifold coordinates. Finding discrete eigenfunctions of the Laplace-Beltrami operator is shown to be equivalent to solving an algebraic eigenvalue problem. The smoothed functions are constructed by…

Face swapping: Automatically replacing faces in photographs

Dmitri Bitouk, Neeraj Kumar, Samreen Dhillon +2 more

Computer Vision and Pattern Recognition · · cited 0× · OpenAlex ↗ · more like this ·

(b) After automatic face replacement Figure 1: We have developed a system that automatically replaces faces in an input image with ones selected from a large collection of face images, obtained by applying face detection to publicly available photographs on the internet. In this example, the faces of (a) two people are shown after (b) automatic replacement with the top three ranked candidates. Our system for face replacement can be used for face de-identification, personalized face replacement, and creating an appealing group photograph from a set of “burst ” mode images. Original images in (a) used with permission from Retna Ltd. (top) and Getty Images Inc. (bottom). In this paper, we present a complete system for automatic face replacement in images. Our system uses a large library of face images created automatically by downloading images from the internet, extracting faces using…

Empirically generated metric spaces for ATR in clutter

Dmitri Bitouk, Michael I. Miller, Laurent Younès

Artificial Intelligence · · cited 0× · DOI ↗ · more like this ·

One of the central problems in automated target recognition is to accommodate the infinite variety of clutter in real military environments. Our principle focus is to construct a metric space where the metric measures the distance between objects of interest invariant to the infinite variety of clutter. Such metrics are formulated using second-order random field models.