Author

Terence Sim

111 papers · 1990–2026

1990peak 11 in 20052026

Frequent co-authors

Papers

showing 1–25 of 111 RSS

The CMU pose, illumination, and expression database

Computer Vision and Pattern Recognition · IEEE Transactions on Pattern Analysis and Machine Intelligence · December 2003 · cited 1,789× · DOI ↗ · more like this ·

In the Fall of 2000, we collected a database of more than 40,000 facial images of 68 people. Using the Carnegie Mellon University 3D Room, we imaged each person across 13 different poses, under 43 different illumination conditions, and with four different expressions. We call this the CMU pose, illumination, and expression (PIE) database. We describe the imaging hardware, the collection procedure, the organization of the images, several possible uses, and how to obtain the database.

The CMU Pose, Illumination, and Expression (PIE) database

Terence Sim, Simon Baker, M. Bsat

Computer Vision and Pattern Recognition · 25 Jun 2003 · cited 1,449× · DOI ↗ · more like this ·

Between October 2000 and December 2000, we collected a database of over 40,000 facial images of 68 people. Using the CMU (Carnegie Mellon University) 3D Room, we imaged each person across 13 different poses, under 43 different illumination conditions, and with four different expressions. We call this database the CMU Pose, Illumination and Expression (PIE) database. In this paper, we describe the imaging hardware, the collection procedure, the organization of the database, several potential uses of the database, and how to obtain the database.

Correlation Filters with Limited Boundaries

Hamed Kiani Galoogahi, Terence Sim, Simon Lucey

cs.CV · 31 Mar 2014 · cited 343× · arXiv ↗ · more like this ·

Correlation filters take advantage of specific properties in the Fourier domain allowing them to be estimated efficiently: O(NDlogD) in the frequency domain, versus O(D^3 + ND^2) spatially where D is signal length, and N is the number of signals. Recent extensions to correlation filters, such as MOSSE, have reignited interest of their use in the vision community due to their robustness and attractive computational properties. In this paper we demonstrate, however, that this computational efficiency comes at a cost. Specifically, we demonstrate that only 1/D proportion of shifted examples are unaffected by boundary effects which has a dramatic effect on detection/tracking performance. In this paper, we propose a novel approach to correlation filter estimation that: (i) takes advantage of inherent computational redundancies in the frequency domain, and (ii) dramatically reduces boundary…

Multi-channel Correlation Filters

Hamed Kiani Galoogahi, Terence Sim, Simon Lucey

Computer Vision and Pattern Recognition · National University of Singapore · December 2013 · cited 272× · DOI ↗ · more like this ·

Modern descriptors like HOG and SIFT are now commonly used in vision for pattern detection within image and video. From a signal processing perspective, this detection process can be efficiently posed as a correlation/ convolution between a multi-channel image and a multi-channel detector/filter which results in a single channel response map indicating where the pattern (e.g. object) has occurred. In this paper, we propose a novel framework for learning a multi-channel detector/filter efficiently in the frequency domain, both in terms of training time and memory footprint, which we refer to as a multichannel correlation filter. To demonstrate the effectiveness of our strategy, we evaluate it across a number of visual detection/ localization tasks where we: (i) exhibit superior performance to current state of the art correlation filters, and (ii) superior computational and memory…

Continuous Verification Using Multimodal Biometrics

Terence Sim, Sheng Zhang, Rajkumar Janakiraman +1 more

Signal Processing · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2 Mar 2007 · cited 223× · DOI ↗ · more like this ·

Conventional verification systems, such as those controlling access to a secure room, do not usually require the user to reauthenticate himself for continued access to the protected resource. This may not be sufficient for high-security environments in which the protected resource needs to be continuously monitored for unauthorized use. In such cases, continuous verification is needed. In this paper, we present the theory, architecture, implementation, and performance of a multimodal biometrics verification system that continuously verifies the presence of a logged-in user. Two modalities are currently used--face and fingerprint--but our theory can be readily extended to include more modalities. We show that continuous verification imposes additional requirements on multimodal fusion when compared to conventional verification systems. We also argue that the usual performance metrics of…

Understanding Humans in Crowded Scenes

Jian Zhao, Jianshu Li, Yu Cheng +3 more

Computer Vision and Pattern Recognition · 15 Oct 2018 · cited 193× · DOI ↗ · more like this ·

Despite the noticeable progress in perceptual tasks like detection, instance segmentation and human parsing, computers still perform unsatisfactorily on visually understanding humans in crowded scenes, such as group behavior analysis, person re-identification and autonomous driving, etc. To this end, models need to comprehensively perceive the semantic information and the differences between instances in a multi-human image, which is recently defined as the multi-human parsing task. In this paper, we present a new large-scale database "Multi-Human Parsing (MHP)" for algorithm development and evaluation, and advances the state-of-the-art in understanding humans in crowded scenes. MHP contains 25,403 elaborately annotated images with 58 fine-grained semantic category labels, involving 2-26 persons per image and captured in real-world scenes from various viewpoints, poses, occlusion…

Understanding Humans in Crowded Scenes: Deep Nested Adversarial Learning and A New Benchmark for Multi-Human Parsing

Jian Zhao, Jianshu Li, Yu Cheng +4 more

cs.CV · April 2018 · cited 193× · arXiv ↗ · more like this ·

Digital face makeup by example

Dong Guo, Terence Sim

Computer Vision and Pattern Recognition · 2009 IEEE Conference on Computer Vision and Pattern Recognition · June 2009 · cited 176× · DOI ↗ · more like this ·

This paper introduces an approach of creating face makeup upon a face image with another image as the style example. Our approach is analogous to physical makeup, as we modify the color and skin detail while preserving the face structure. More precisely, we first decompose the two images into three layers: face structure layer, skin detail layer, and color layer. Thereafter, we transfer information from each layer of one image to corresponding layer of the other image. One major advantage of the proposed method lies in that only one example image is required. This renders face makeup by example very convenient and practical. Equally, this enables some additional interesting applications, such as applying makeup by a portraiture. The experiment results demonstrate the effectiveness of the proposed approach in faithfully transferring makeup.

Enhancing photographs with Near Infra-Red images

Xiaopeng Zhang, Terence Sim, Xiaoping Miao

Computer Vision and Pattern Recognition · National University of Singapore · June 2008 · cited 109× · DOI ↗ · more like this ·

Near Infra-Red (NIR) images of natural scenes usually have better contrast and contain rich texture details that may not be perceived in visible light photographs (VIS). In this paper, we propose a novel method to enhance a photograph by using the contrast and texture information of its corresponding NIR image. More precisely, we first decompose the NIR/VIS pair into average and detail wavelet subbands. We then transfer the contrast in the average subband and transfer texture in the detail subbands. We built a special camera mount that optically aligns two consumer-grade digital cameras, one of which was modified to capture NIR. Our results exhibit higher visual quality than tone-mapped HDR images, showing that NIR imaging is useful for computational photography.

Discriminant Subspace Analysis: A Fukunaga-Koontz Approach

Sheng Zhang, Terence Sim

Computer Vision and Pattern Recognition · IEEE Transactions on Pattern Analysis and Machine Intelligence · 29 Aug 2007 · cited 100× · DOI ↗ · more like this ·

The Fisher Linear Discriminant (FLD) is commonly used in pattern recognition. It finds a linear subspace that maximally separates class patterns according to the Fisher Criterion. Several methods of computing the FLD have been proposed in the literature, most of which require the calculation of the so-called scatter matrices. In this paper, we bring a fresh perspective to FLD via the Fukunaga-Koontz Transform (FKT). We do this by decomposing the whole data space into four subspaces with different discriminability, as measured by eigenvalue ratios. By connecting the eigenvalue ratio with the generalized eigenvalue, we show where the Fisher Criterion is maximally satisfied. We prove the relationship between FLD and FKT analytically, and propose a unified framework to understanding some existing work. Furthermore, we extend our our theory to Multiple Discriminant Analysis (MDA). This is…

Are Digraphs Good for Free-Text Keystroke Dynamics?

Terence Sim, Rajkumar Janakiraman

Information Systems · National University of Singapore · June 2007 · cited 86× · DOI ↗ · more like this ·

Research in keystroke dynamics has largely focused on the typing patterns found in fixed text (e.g. userid and passwords). In this regard, digraphs and trigraphs have proven to be discriminative features. However, there is increasing interest in free-text keystroke dynamics, in which the user to be authenticated is free to type whatever he/she wants, rather than a pre-determined text. The natural question that arises is whether digraphs and trigraphs are just as discriminative for free text as they are for fixed text. We attempt to answer this question in this paper. We show that digraphs and trigraphs, if computed without regard to what word was typed, are no longer discriminative. Instead, word-specific digraphs/trigraphs are required. We also show that the typing dynamics for some words depend on whether they are part of a larger word. Our study is the first to investigate these…

Correcting over-exposure in photographs

Dong Guo, Yuan Cheng, Shaojie Zhuo +1 more

Computer Vision and Pattern Recognition · National University of Singapore · June 2010 · cited 82× · DOI ↗ · more like this ·

This paper introduces a method to correct over-exposure in an existing photograph by recovering the color and lightness separately. First, the dynamic range of well exposed region is slightly compressed to make room for the recovered lightness of the over-exposed region. Then the lightness is recovered based on an over-exposure likelihood. The color of each pixel is corrected via neighborhood propagation and also based on the confidence of the original color. Previous methods make use of ratios between different color channels to recover the over-exposed ones, and thus can not handle regions where all three channels are over-exposed. In contrast, our method does not have this limitation. Our method is fully automatic and requires only one single input photo. We also provide users with the flexibility to control the amount of over-exposure correction. Experiment results demonstrate the…

Enhancing low light images using near infrared flash images

Shaojie Zhuo, Xiaopeng Zhang, Xiaoping Miao +1 more

Computer Vision and Pattern Recognition · National University of Singapore · September 2010 · cited 79× · DOI ↗ · more like this ·

In low light environment, photographs taken with a high ISO setting suffer from significant noise. In this paper, we propose to use a near infrared (NIR) flash image, instead of a normal visible flash image, to enhance its corresponding noisy visible image. We build a hybrid camera system to take an visible image and its NIR counterpart simultaneously. We introduce a new method to denoise an visible image and enhance its details using its corresponding NIR flash image. Experimental results show the superiority of our method compared with previous image denoising and detail enhancement methods.

Inter-modality Face Sketch Recognition

Hamed Kiani Galoogahi, Terence Sim

Computer Vision and Pattern Recognition · National University of Singapore · July 2012 · cited 72× · DOI ↗ · more like this ·

Automatic face sketch recognition plays an important role in law enforcement. Recently, various methods have been proposed to address the problem of face sketch recognition by matching face photos and sketches, which are of different modalities. However, their performance is strongly affected by the modality difference between sketches and photos. In this paper, we propose a new face descriptor based on gradient orientations to reduce the modality difference in feature extraction stage, called Histogram of Averaged Oriented Gradients (HAOG). Experiments on CUFS database show that the new descriptor outperforms the state-of-the-art approaches.

Memory-based face recognition for visitor identification

Terence Sim, Rahul Sukthankar, Matthew D. Mullin +1 more

Computer Vision and Pattern Recognition · 7 Nov 2002 · cited 69× · DOI ↗ · more like this ·

We show that a simple, memory-based technique for appearance-based face recognition, motivated by the real-world task of visitor identification, can outperform more sophisticated algorithms that use principal components analysis (PCA) and neural networks. This technique is closely related to correlation templates; however, we show that the use of novel similarity measures greatly improves performance. We also show that augmenting the memory base with additional, synthetic face images results in further improvements in performance. Results of extensive empirical testing on two standard face recognition datasets are presented, and direct comparisons with published work show that our algorithm achieves comparable (or superior) results. Our system is incorporated into an automated visitor identification system that has been operating successfully in an outdoor environment since January 1999.

Face sketch recognition by Local Radon Binary Pattern: LRBP

Hamed Kiani Galoogahi, Terence Sim

Computer Vision and Pattern Recognition · National University of Singapore · September 2012 · cited 61× · DOI ↗ · more like this ·

In this paper, we propose a new face descriptor to directly match face photos and sketches of different modalities, called Local Radon Binary Pattern (LRBP). LRBP is inspired by the fact that the shape of a face photo and its corresponding sketch is similar, even when the sketch is exaggerated by an artist. Therefore, the shape of face can be exploited to compute features which are robust against modality differences between face photo and sketch. In LRBP framework, the characteristics of face shape are captured by transforming face image into Radon space. Then, micro-information of face shape in new space is encoded by Local Binary Pattern (LBP). Finally, LRBP is computed by concatenating histograms of local LBPs. In order to capture both local and global characteristics of face shape, LRBP is extracted in a spatial pyramid fashion. Experiments on CUFS and CUFSF datasets indicate the…

Learning universal multi-view age estimator using video context

Zheng Song, Bingbing Ni, Dong Guo +2 more

Computer Vision and Pattern Recognition · National University of Singapore · November 2011 · cited 57× · DOI ↗ · more like this ·

Many existing techniques for analyzing face images assume that the faces are at nearly frontal. Generalizing to non-frontal faces is often difficult, due to a dearth of ground truth for non-frontal faces and also to the inherent challenges in handling pose variations. In this work, we investigate how to learn a universal multi-view age estimator by harnessing 1) unlabeled web videos, 2) a publicly available labeled frontal face corpus, and 3) zero or more non-frontal faces with age labels. First, a large diverse human-involved video corpus is collected from online video sharing website. Then, multi-view face detection and tracking are performed to build a large set of frontal-vs-profile face bundles, each of which is from the same tracking sequence, and thus exhibiting the same age. These unlabeled face bundles constitute the so-called video context, and the parametric multi-view age…

Wonder Ears: Identification of Identical Twins from Ear Images

Hossein Nejati, Li Zhang, Terence Sim +2 more

Signal Processing · National University of Singapore · 2012 · cited 55× · OpenAlex ↗ · more like this ·

While identical twins identification is a well known challenge in face recognition, it seems that no work has explored automatic ear recognition for identical twin identification. Ear image recognition has been stud-ied for years, but Iannarelli (1989) appears to be the only work mentioning the twin identification (performed manually). We here explore the possibility of automatic twin identification from their ear images based on a recently proposed psychological model for face recog-nition in humans, known as Exception Report Model (ERM). We test our new approach on 39 pairs of identi-cal twins (78 subjects), with several levels of resolution, occlusion and noise, left ear vs. right ear, and feature optimization which verifies the robustness of the intro-duced features. 1

Robust Face Recognition with Deep Multi-View Representation Learning

Jianshu Li, Jian Zhao, Fang Zhao +5 more

Computer Vision and Pattern Recognition · 29 Sep 2016 · cited 50× · DOI ↗ · more like this ·

This paper describes our proposed method targeting at the MSR Image Recognition Challenge MS-Celeb-1M. The challenge is to recognize one million celebrities from their face images captured in the real world. The challenge provides a large scale dataset crawled from the Web, which contains a large number of celebrities with many images for each subject. Given a new testing image, the challenge requires an identify for the image and the corresponding confidence score. To complete the challenge, we propose a two-stage approach consisting of data cleaning and multi-view deep representation learning. The data cleaning can effectively reduce the noise level of training data and thus improves the performance of deep learning based face recognition models. The multi-view representation learning enables the learned face representations to be more specific and discriminative. Thus the…

Happiness level prediction with sequential inputs via multiple regressions

Jianshu Li, Sujoy Roy, Jiashi Feng +1 more

Computer Vision and Pattern Recognition · 31 Oct 2016 · cited 48× · DOI ↗ · more like this ·

This paper presents our solution submitted to the Emotion Recognition in the Wild (EmotiW 2016) group-level happiness intensity prediction sub-challenge. The objective of this sub-challenge is to predict the overall happiness level given an image of a group of people in a natural setting. We note that both the global setting and the faces of the individuals in the image influence the group-level happiness intensity of the image. Hence the challenge lies in building a solution that incorporates both these factors and also considers their right combination. Our proposed solution incorporates both these factors as a combination of global and local information. We use a convolutional neural network to extract discriminative face features, and a recurrent neural network to selectively memorize the important features to perform the group-level happiness prediction task. Experimental…

Using Continuous Biometric Verification to Protect Interactive Login Sessions

Sandeep Kumar, Terence Sim, Rajkumar Janakiraman +1 more

Signal Processing · National University of Singapore · 5 Jan 2006 · cited 43× · DOI ↗ · more like this ·

In this paper we describe the theory, architecture, implementation, and performance of a multimodal passive biometric verification system that continually verifies the presence/participation of a logged-in user. We assume that the user logged in using strong authentication prior to the starting of the continuous verification process. While the implementation described in the paper combines a digital camera-based face verification with a mouse-based fingerprint reader, the architecture is generic enough to accommodate additional biometric devices with different accuracy of classifying a given user from an imposter. The main thrust of our work is to build a multimodal biometric feedback mechanism into the operating system so that verification failure can automatically lock up the computer within some estimate of the time it takes to subvert the computer. This must be done with low false…

Face Anonymization Made Simple

Han-Wei Kung, Tuomas Varanka, Sanjay Saha +2 more

cs.CV · November 2024 · cited 39× · arXiv ↗ · more like this ·

Current face anonymization techniques often depend on identity loss calculated by face recognition models, which can be inaccurate and unreliable. Additionally, many methods require supplementary data such as facial landmarks and masks to guide the synthesis process. In contrast, our approach uses diffusion models with only a reconstruction loss, eliminating the need for facial landmarks or masks while still producing images with intricate, fine-grained details. We validated our results on two public benchmarks through both quantitative and qualitative evaluations. Our model achieves state-of-the-art performance in three key areas: identity anonymization, facial attribute preservation, and image quality. Beyond its primary function of anonymization, our model can also perform face swapping tasks by incorporating an additional facial image as input, demonstrating its versatility and…

Automatic acquisition of exemplar-based representations for recognition from image sequences

Christian Wallraven, HH Bülthoff, Terence Sim

Computer Vision and Pattern Recognition · MPG.PuRe (Max Planck Society) · December 2001 · cited 33× · OpenAlex ↗ · more like this ·

We present an exemplar-based object recognition system which is capable of on-line learning of representations of scenes and objects from image sequences. Local appearance features are used in a tracking framework to find `key-frames' of the input sequence during learning. The representation of the stored sequences which are used for recognition of novel images consists only of the appearance features in these key-frames and contains no further a-priori assumptions about the underlying sequences. The system is able to create sparse and extendable representations and shows good recognition performance in a variety of viewing conditions for databases of natural and synthetic image sequences.

Landmark Free Face Attribute Prediction

Jianshu Li, Fang Zhao, Jiashi Feng +3 more

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · 21 May 2018 · cited 32× · DOI ↗ · more like this ·

Face attribute prediction in the wild is important for many facial analysis applications yet it is very challenging due to ubiquitous face variations. In this paper, we address face attribute prediction in the wild by proposing a novel method, lAndmark Free Face AttrIbute pRediction (AFFAIR). Unlike traditional face attribute prediction methods that require facial landmark detection and face alignment, AFFAIR uses an endto- end learning pipeline to jointly learn a hierarchy of spatial transformations that optimize facial attribute prediction with no reliance on landmark annotations or pre-trained landmark detectors. AFFAIR achieves this through simultaneously 1) learning a global transformation which effectively alleviates negative effect of global face variation for the following attribute prediction tailored for each face, 2) locating the most relevant facial part for attribute…

Toward a Comprehensive Face Detector in the Wild

Jianshu Li, Luoqi Liu, Jianan Li +3 more

Computer Vision and Pattern Recognition · IEEE Transactions on Circuits and Systems for Video Technology · 28 Nov 2017 · cited 32× · DOI ↗ · more like this ·

In this paper, we aim to build a comprehensive face detection system which provides a one-stop solution to various practical challenges for face detection in realistic scenarios, e.g., detecting faces from multiple-views, faces with occlusions, exaggerated expressions or blurred faces. Moreover, we introduce an automatic data harvest algorithm to effectively improve the generalization performance of the system even when collecting training faces containing various challenging patterns is difficult. In particular, we introduce three critical components to build the system, i.e., a recently widely used deep convolutional neural network (CNN), a novel blur-aware bi-channel network architecture, and a new self-learning mechanism capable of exploiting video contexts continuously. The aforementioned challenges except for detecting blurred faces can potentially be addressed by the CNN…