Author

Matthew Turk

222 papers · 1985–2026

1985peak 14 in 20042026

Frequent co-authors

Papers

showing 1–25 of 222 RSS

Eigenfaces for Recognition

Computer Vision and Pattern Recognition · Journal of Cognitive Neuroscience · 1991 · cited 13,775× · DOI ↗ · more like this ·

We have developed a near-real-time computer system that can locate and track a subject's head, and then recognize the person by comparing characteristics of the face to those of known individuals. The computational approach taken in this system is motivated by both physiology and information theory, as well as by the practical requirements of near-real-time performance and accuracy. Our approach treats the face recognition problem as an intrinsically two-dimensional (2-D) recognition problem rather than requiring recovery of three-dimensional geometry, taking advantage of the fact that faces are normally upright and thus may be described by a small set of 2-D characteristic views. The system functions by projecting face images onto a feature space that spans the significant variations among known face images. The significant features are known as "eigenfaces," because they are the…

Face recognition using eigenfaces

Matthew Turk, Alex Pentland

Computer Vision and Pattern Recognition · 10 Dec 2002 · cited 5,211× · DOI ↗ · more like this ·

An approach to the detection and identification of human faces is presented, and a working, near-real-time face recognition system which tracks a subject's head and then recognizes the person by comparing characteristics of the face to those of known individuals is described. This approach treats face recognition as a two-dimensional recognition problem, taking advantage of the fact that faces are normally upright and thus may be described by a small set of 2-D characteristic views. Face images are projected onto a feature space ('face space') that best encodes the variation among known face images. The face space is defined by the 'eigenfaces', which are the eigenvectors of the set of faces; they do not necessarily correspond to isolated features such as eyes, ears, and noses. The framework provides the ability to learn to recognize new faces in an unsupervised manner.< <ETX…

Transformed Social Interaction: Decoupling Representation from Behavior and Form in Collaborative Virtual Environments

Jeremy N. Bailenson, Andrew C. Beall, Jack M. Loomis +2 more

Human-Computer Interaction · PRESENCE Virtual and Augmented Reality · August 2004 · cited 300× · DOI ↗ · more like this ·

Computer-mediated communication systems known as collaborative virtual environments (CVEs) allow geographically separated individuals to interact verbally and nonverbally in a shared virtual space in real time. We discuss a CVE-based research paradigm that transforms (i.e., filters and modifies) nonverbal behaviors during social interaction. Because the technology underlying CVEs allows a strategic decoupling of rendered behavior from the actual behavior of the interactants, conceptual and perceptual constraints inherent in face-to-face interaction need not apply. Decoupling algorithms can enhance or degrade facets of nonverbal behavior within CVEs, such that interactants can reap the benefits of nonverbal enhancement or suffer nonverbal degradation. Concepts underlying transformed social interaction (TSI), the ethics and implications of such a research paradigm, and data from a pilot…

Robust hand detection

Mathias Kölsch, Matthew Turk

Human-Computer Interaction · 10 Jun 2004 · cited 268× · DOI ↗ · more like this ·

Vision-based hand gesture interfaces require fast and extremely robust hand detection. Here, we study view-specific hand posture detection with an object recognition method proposed by Viola and Jones. Training with this method is computationally very expensive, prohibiting the evaluation of many hand appearances for their suitability to detection. In this paper, we present a frequency analysis-based method for instantaneous estimation of class separability, without the need for any training. We built detectors for the most promising candidates, their receiver operating characteristics confirming the estimates. Next, we found that classification accuracy increases with a more expressive feature type. Lastly, we show that further optimization of training parameters yields additional detection rate improvements. In summary, we present a systematic approach to building an extremely robust…

Gesture modeling and recognition using finite state machines

Pengyu Hong, Matthew Turk, Thomas S. Huang

Human-Computer Interaction · 7 Nov 2002 · cited 217× · DOI ↗ · more like this ·

We propose a state-based approach to gesture learning and recognition. Using spatial clustering and temporal alignment, each gesture is defined to be an ordered sequence of states in spatial-temporal space. The 2D image positions of the centers of the head and both hands of the user are used as features; these are located by a color-based tracking method. From training data of a given gesture, we first learn the spatial information and then group the data into segments that are automatically aligned temporally. The temporal information is further integrated to build a finite state machine (FSM) recognizer. Each gesture has a FSM corresponding to it. The computational efficiency of the FSM recognizers allows us to achieve real-time on-line performance. We apply this technique to build an experimental system that plays a game of "Simon Says" with the user.

Effective representation using ICA for face recognition robust to local distortion and partial occlusion

Jongsun Kim, Jongmoo Choi, Juneho Yi +1 more

Signal Processing · IEEE Transactions on Pattern Analysis and Machine Intelligence · November 2005 · cited 215× · DOI ↗ · more like this ·

The performance of face recognition methods using subspace projection is directly related to the characteristics of their basis images, especially in the cases of local distortion or partial occlusion. In order for a subspace projection method to be robust to local distortion and partial occlusion, the basis images generated by the method should exhibit a part-based local representation. We propose an effective part-based local representation method named locally salient ICA (LS-ICA) method for face recognition that is robust to local distortion and partial occlusion. The LS-ICA method only employs locally salient information from important facial parts in order to maximize the benefit of applying the idea of "recognition by parts." It creates part-based local basis images by imposing additional localization constraint in the process of computing ICA architecture I basis images. We have…

Non-photorealistic camera

Ramesh Raskar, Kar-Han Tan, Rogério Feris +2 more

Computer Vision and Pattern Recognition · ACM Transactions on Graphics · August 2004 · cited 211× · DOI ↗ · more like this ·

We present a non-photorealistic rendering approach to capture and convey shape features of real-world scenes. We use a camera with multiple flashes that are strategically positioned to cast shadows along depth discontinuities in the scene. The projective-geometric relationship of the camera-flash setup is then exploited to detect depth discontinuities and distinguish them from intensity edges due to material discontinuities.We introduce depiction methods that utilize the detected edge features to generate stylized static and animated images. We can highlight the detected features, suppress unnecessary details or combine features from multiple images. The resulting images more clearly convey the 3D structure of the imaged scenes.We take a very different approach to capturing geometric features of a scene than traditional approaches that require reconstructing a 3D model. This results in…

View-based interpretation of real-time optical flow for gesture recognition

Ross Cutler, Matthew Turk

Computer Vision and Pattern Recognition · 27 Nov 2002 · cited 208× · DOI ↗ · more like this ·

The authors have developed a real-time, view-based gesture recognition system. Optical flow is estimated and segmented into motion blobs. Gestures are recognized using a rule-based technique based on characteristics of the motion blobs such as relative motion and size. Parameters of the gesture (e.g., frequency) are then estimated using context specific techniques. The system has been applied to create an interactive environment for children.

Attribute-based people search in surveillance environments

Daniel Vaquero, Rogério Feris, Duan Tran +3 more

Computer Vision and Pattern Recognition · December 2009 · cited 197× · DOI ↗ · more like this ·

We propose a novel framework for searching for people in surveillance environments. Rather than relying on face recognition technology, which is known to be sensitive to typical surveillance conditions such as lighting changes, face pose variation, and low-resolution imagery, we approach the problem in a different way: we search for people based on a parsing of human parts and their attributes, including facial hair, eyewear, clothing color, etc. These attributes can be extracted using detectors learned from large amounts of training data. A complete system that implements our framework is presented. At the interface, the user can specify a set of personal characteristics, and the system then retrieves events that match the provided description. For example, a possible query is ¿show me the bald people who entered a given building last Saturday wearing a red shirt and sunglasses.¿ This…

World-stabilized annotations and virtual scene navigation for remote collaboration

Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk +1 more

Computer Vision and Pattern Recognition · October 2014 · cited 191× · DOI ↗ · more like this ·

We present a system that supports an augmented shared visual space for live mobile remote collaboration on physical tasks. The remote user can explore the scene independently of the local user's current camera position and can communicate via spatial annotations that are immediately visible to the local user in augmented reality. Our system operates on off-the-shelf hardware and uses real-time visual tracking and modeling, thus not requiring any preparation or instrumentation of the environment. It creates a synergy between video conferencing and remote scene exploration under a unique coherent interface. To evaluate the collaboration with our system, we conducted an extensive outdoor user study with 60 participants comparing our system with two baseline interfaces. Our results indicate an overwhelming user preference (80%) for our system, a high level of usability, as well as…

Transformed Social Interaction, Augmented Gaze, and Social Influence in Immersive Virtual Environments

Jeremy N. Bailenson, Andrew C. Beall, Jack M. Loomis +2 more

Human-Computer Interaction · Human Communication Research · October 2005 · cited 186× · DOI ↗ · more like this ·

Immersive collaborative virtual environments (CVEs) are simulations in which geographically separated individuals interact in a shared, three-dimensional, digital space using immersive virtual environment technology. Unlike videoconference technology, which transmits direct video streams, immersive CVEs accurately track movements of interactants and render them nearly simultaneously (i.e., in real time) onto avatars, three-dimensional digital representations of the interactants. Nonverbal behaviors of interactants can be rendered veridically or transformed strategically (i.e., rendered nonveridically). This research examined augmented gaze, a transformation in which a given interactant's actual head movements are transformed by an algorithm that renders his or her gaze directly at multiple interactants simultaneously, such that each of the others perceives that the transformed…

Fast 2D Hand Tracking with Flocks of Features and Multi-Cue Integration

Mathias Kölsch, Matthew Turk

Computer Vision and Pattern Recognition · April 2005 · cited 184× · DOI ↗ · more like this ·

This paper introduces "Flocks of Features," a fast tracking method for non-rigid and highly articulated objects such as hands. It combines KLT features and a learned foreground color distribution to facilitate 2D position tracking from a monocular view. The tracker's benefits lie in its speed, its robustness against background noise, and its ability to track objects that undergo arbitrary rotations and vast and rapid deformations. We demonstrate tracker performance on hand tracking with a non-stationary camera in unconstrained indoor and outdoor environments. The tracker yields over threefold improvement over a CamShift tracker in terms of the number of frames tracked before the target was lost, and often more than one order of magnitude improvement in terms of the fractions of particular test sequences tracked successfully.

Perceptual user interfaces (introduction)

Matthew Turk, George Robertson

Human-Computer Interaction · Communications of the ACM · March 2000 · cited 163× · DOI ↗ · more like this ·

<html xmlns:o=3D"urn:schemas-microsoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" xmlns=3D"http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv=3DContent-Type content=3D"text/html; charset=3Dwindows-1252"> <meta name=3DProgId content=3DWord.Document> <meta name=3DGenerator content=3D"Microsoft Word 11"> <meta name=3DOriginator content=3D"Microsoft Word 11"> <link rel=3DFile-List href=3D"The%20article%20focuses%20on%20user%20interfaces_archivos/filelist.xml"> <title>The article focuses on user interfaces</title> <!--[if gte mso 9]><xml> <o:DocumentProperties> <o:Author>Biblioteca de Ciencias y Tecnología</o:Author> <o:Template>Normal</o:Template> <o:LastAuthor>Biblioteca de Ciencias y Tecnología</o:LastAuthor> <o:Revision>2</o:Revision> <o:TotalTime>1</o:TotalTime> <o:Created>2006-05-24T23:02:00Z</o:Created> <o:LastSaved>2006-05-24T23:02:00Z</o:LastSaved>…

Optimizing the Viewing Graph for Structure-from-Motion

Chris Sweeney, Torsten Sattler, Tobias Höllerer +2 more

Computer Vision and Pattern Recognition · December 2015 · cited 134× · DOI ↗ · more like this ·

The viewing graph represents a set of views that are related by pairwise relative geometries. In the context of Structure-from-Motion (SfM), the viewing graph is the input to the incremental or global estimation pipeline. Much effort has been put towards developing robust algorithms to overcome potentially inaccurate relative geometries in the viewing graph during SfM. In this paper, we take a fundamentally different approach to SfM and instead focus on improving the quality of the viewing graph before applying SfM. Our main contribution is a novel optimization that improves the quality of the relative geometries in the viewing graph by enforcing loop consistency constraints with the epipolar point transfer. We show that this optimization greatly improves the accuracy of relative poses in the viewing graph and removes the need for filtering steps or robust algorithms typically used in…

Integrating the physical environment into mobile remote collaboration

Steffen Gauglitz, Cha Lee, Matthew Turk +1 more

Computer Vision and Pattern Recognition · 21 Sep 2012 · cited 133× · DOI ↗ · more like this ·

We describe a framework and prototype implementation for unobtrusive mobile remote collaboration on tasks that involve the physical environment. Our system uses the Augmented Reality paradigm and model-free, markerless visual tracking to facilitate decoupled, live updated views of the environment and world-stabilized annotations while supporting a moving camera and unknown, unprepared environments. In order to evaluate our concept and prototype, we conducted a user study with 48 participants in which a remote expert instructed a local user to operate a mock-up airplane cockpit. Users performed significantly better with our prototype (40.8 tasks completed on average) as well as with static annotations (37.3) than without annotations (28.9). 79% of the users preferred our prototype despite noticeably imperfect tracking.

Probabilistic expression analysis on manifolds

Ya Chang, Changbo Hu, Matthew Turk

Computer Vision and Pattern Recognition · 12 Nov 2004 · cited 123× · DOI ↗ · more like this ·

In this paper, we propose a probabilistic video-based facial expression recognition method on manifolds. The concept of the manifold of facial expression is based on the observation that the images of all possible facial deformations of an individual make a smooth manifold embedded in a high dimensional image space. An enhanced Lipschitz embedding is developed to embed the aligned face appearance in a low dimensional space while keeping the main structure of the manifold. In the embedded space, a complete expression sequence becomes a path on the expression manifold, emanating from a center that corresponds to the neutral expression. Each path consists of several clusters. A probabilistic model of transition between the clusters and paths is learned through training videos in the embedded space. The likelihood of one kind of facial expression is modeled as a mixture density with the…

Manifold of facial expression

Yi-Ting Chang, Chaofang Hu, Matthew Turk

Computer Vision and Pattern Recognition · 23 Apr 2004 · cited 121× · DOI ↗ · more like this ·

We propose the concept of manifold of facial expression based on the observation that images of a subject's facial expressions define a smooth manifold in the high dimensional image space. Such a manifold representation can provide a unified framework for facial expression analysis. We first apply active wavelet networks (AWN) on the image sequences for facial feature localization. To learn the structure of the manifold in the feature space derived by AWN, we investigated two types of embeddings from a high dimensional space to a low dimensional space: locally linear embedding (LLE) and Lipschitz embedding. Our experiments show that LLE is suitable for visualizing expression manifolds. After applying Lipschitz embedding, the expression manifold can be approximately considered as a super-spherical surface in the embedding space. For manifolds derived from different subjects, we propose a…

A survey of image retargeting techniques

Daniel Vaquero, Matthew Turk, Kari Pulli +2 more

Computer Vision and Pattern Recognition · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 19 Aug 2010 · cited 118× · DOI ↗ · more like this ·

Advances in imaging technology have made the capture and display of digital images ubiquitous. A variety of displays are used to view them, ranging from high-resolution computer monitors to low-resolution mobile devices, and images often have to undergo changes in size and aspect ratio to adapt to different screens. Also, displaying and printing documents with embedded images frequently entail resizing of the images to comply with the overall layout. Straightforward image resizing operators, such as scaling, often do not produce satisfactory results, since they are oblivious to image content. In this work, we review and categorize algorithms for contentaware image retargeting, i.e., resizing an image while taking its content into consideration to preserve important regions and minimize distortions. This is a challenging problem, as it requires preserving the relevant information while…

In touch with the remote world

Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk +1 more

Computer Vision and Pattern Recognition · 7 Nov 2014 · cited 114× · DOI ↗ · more like this ·

Augmented reality annotations and virtual scene navigation add new dimensions to remote collaboration. In this paper, we present a touchscreen interface for creating freehand drawings as world-stabilized annotations and for virtually navigating a scene reconstructed live in 3D, all in the context of live remote collaboration. Two main focuses of this work are (1) automatically inferring depth for 2D drawings in 3D space, for which we evaluate four possible alternatives, and (2) gesture-based virtual navigation designed specifically to incorporate constraints arising from partially modeled remote scenes. We evaluate these elements via qualitative user studies, which in addition provide insights regarding the design of individual visual feedback elements and the need to visualize the direction of drawings.

NON-NEGATIVE MATRIX FACTORIZATION FRAMEWORK FOR FACE RECOGNITION

Yuan Wang, Yunde Jia, Changbo Hu +1 more

Computer Vision and Pattern Recognition · International Journal of Pattern Recognition and Artificial Intelligence · June 2005 · cited 105× · DOI ↗ · more like this ·

Non-negative Matrix Factorization (NMF) is a part-based image representation method which adds a non-negativity constraint to matrix factorization. NMF is compatible with the intuitive notion of combining parts to form a whole face. In this paper, we propose a framework of face recognition by adding NMF constraint and classifier constraints to matrix factorization to get both intuitive features and good recognition results. Based on the framework, we present two novel subspace methods: Fisher Non-negative Matrix Factorization (FNMF) and PCA Non-negative Matrix Factorization (PNMF). FNMF adds both the non-negative constraint and the Fisher constraint to matrix factorization. The Fisher constraint maximizes the between-class scatter and minimizes the within-class scatter of face samples. Subsequently, FNMF improves the capability of face recognition. PNMF adds the non-negative constraint…

A Random Walk through Eigenspace

Matthew Turk

Computer Vision and Pattern Recognition · IEICE Transactions on Information and Systems · December 2001 · cited 98× · OpenAlex ↗ · more like this ·

SUMMARY It has been over a decade since the “Eigenfaces” approach to automatic face recognition, and other appearancebased methods, made an impression on the computer vision research community and helped spur interest in vision systems being used to support biometrics and human-computer interface. In this paper I give a personal viewof the original motivation for the work, some of the strengths and limitation of the approach, and progress in the years since. Appearance-based approaches to recognition complement feature- or shape-based approaches, and a practical face recognition system should have elements of both. Eigenfaces is not a general approach to recognition, but rather one tool out of many to be applied and evaluated in the appropriate context.

TranslatAR: A mobile augmented reality translator

Victor Fragoso, Steffen Gauglitz, Shane Zamora +2 more

Computer Vision and Pattern Recognition · 2011 · cited 94× · DOI ↗ · more like this ·

We present a mobile augmented reality (AR) translation system, using a smartphone's camera and touchscreen, that requires the user to simply tap on the word of interest once in order to produce a translation, presented as an AR overlay. The translation seamlessly replaces the original text in the live camera stream, matching background and foreground colors estimated from the source images. For this purpose, we developed an efficient algorithm for accurately detecting the location and orientation of the text in a live camera stream that is robust to perspective distortion, and we combine it with OCR and a text-to-text translation engine. Our experimental results, using the ICDAR 2003 dataset and our own set of video sequences, quantify the accuracy of our detection and analyze the sources of failure among the system's components. With the OCR and translation running in a background…

EVSAC: Accelerating Hypotheses Generation by Modeling Matching Scores with Extreme Value Theory

Victor Fragoso, Pradeep Sen, Sergio Rodriguez +1 more

Computer Vision and Pattern Recognition · December 2013 · cited 91× · DOI ↗ · more like this ·

Algorithms based on RANSAC that estimate models using feature correspondences between images can slow down tremendously when the percentage of correct correspondences (inliers) is small. In this paper, we present a probabilistic parametric model that allows us to assign confidence values for each matching correspondence and therefore accelerates the generation of hypothesis models for RANSAC under these conditions. Our framework leverages Extreme Value Theory to accurately model the statistics of matching scores produced by a nearest-neighbor feature matcher. Using a new algorithm based on this model, we are able to estimate accurate hypotheses with RANSAC at low inlier ratios significantly faster than previous state-of-the-art approaches, while still performing comparably when the number of inliers is large. We present results of homography and fundamental matrix estimation experiments…

Theia

Christopher J. Sweeney, Tobias Höllerer, Matthew Turk

Computer Vision and Pattern Recognition · 13 Oct 2015 · cited 86× · DOI ↗ · more like this ·

In this paper, we have presented a comprehensive multi-view geometry library, Theia, that focuses on large-scale SfM. In addition to state-of-the-art scalable SfM pipelines, the library provides numerous tools that are useful for students, researchers, and industry experts in the field of multi-view geometry. Theia contains clean code that is well documented (with code comments and the website) and easy to extend. The modular design allows for users to easily implement and experiment with new algorithms within our current pipeline without having to implement a full end-to-end SfM pipeline themselves. Theia has already gathered a large number of diverse users from universities, startups, and industry and we hope to continue to gather users and active contributors from the open-source community.

Computer vision in the interface

Matthew Turk

Computer Vision and Pattern Recognition · Communications of the ACM · 2004 · cited 84× · DOI ↗ · more like this ·

There are still obstacles to achieving general, robust, high-performance computer vision systems. The last decade, however, has seen significant progress in vision technologies for human-computer interaction.