Author

Hu Han

167 papers · 1999–2026

1999peak 16 in 20192026

Frequent co-authors

Papers

showing 1–25 of 167 RSS

Face Spoof Detection With Image Distortion Analysis

Signal Processing · IEEE Transactions on Information Forensics and Security · 4 Feb 2015 · cited 911× · DOI ↗ · more like this ·

Automatic face recognition is now widely used in applications ranging from deduplication of identity to authentication of mobile payment. This popularity of face recognition has raised concerns about face spoof attacks (also known as biometric sensor presentation attacks), where a photo or video of an authorized person's face could be used to gain access to facilities or services. While a number of face spoof detection techniques have been proposed, their generalization ability has not been adequately addressed. We propose an efficient and rather robust face spoof detection algorithm based on image distortion analysis (IDA). Four different features (specular reflection, blurriness, chromatic moment, and color diversity) are extracted to form the IDA feature vector. An ensemble classifier, consisting of multiple SVM classifiers trained for different face spoof attacks (e.g., printed…

Secure Face Unlock: Spoof Detection on Smartphones

Keyurkumar J. Patel, Hu Han, Anil K. Jain

Signal Processing · IEEE Transactions on Information Forensics and Security · 8 Jun 2016 · cited 517× · DOI ↗ · more like this ·

With the wide deployment of the face recognition systems in applications from deduplication to mobile device unlocking, security against the face spoofing attacks requires increased attention; such attacks can be easily launched via printed photos, video replays, and 3D masks of a face. We address the problem of face spoof detection against the print (photo) and replay (photo or video) attacks based on the analysis of image distortion (e.g., surface reflection, moiré pattern, color distortion, and shape deformation) in spoof face images (or video frames). The application domain of interest is smartphone unlock, given that the growing number of smartphones have the face unlock and mobile payment capabilities. We build an unconstrained smartphone spoof attack database (MSU USSA) containing more than 1000 subjects. Both the print and replay attacks are captured using the front and rear…

Automated Systolic Array Architecture Synthesis for High Throughput CNN Inference on FPGAs

Xuechao Wei, Cody Hao Yu, Peng Zhang +5 more

Computer Vision and Pattern Recognition · 13 Jun 2017 · cited 416× · DOI ↗ · more like this ·

Convolutional neural networks (CNNs) have been widely applied in many deep learning applications. In recent years, the FPGA implementation for CNNs has attracted much attention because of its high performance and energy efficiency. However, existing implementations have difficulty to fully leverage the computation power of the latest FPGAs. In this paper we implement CNN on an FPGA using a systolic array architecture, which can achieve high clock frequency under high resource utilization. We provide an analytical model for performance and resource utilization and develop an automatic design space exploration framework, as well as source-to-source code transformation from a C program to a CNN implementation using systolic array. The experimental results show that our framework is able to generate the accelerator for real-life CNN models, achieving up to 461 GFlops for floating point data…

SeqTrack: Sequence to Sequence Learning for Visual Object Tracking

Xin Chen, Houwen Peng, Dong Wang +2 more

Computer Vision and Pattern Recognition · June 2023 · cited 393× · DOI ↗ · more like this ·

In this paper, we present a new sequence-to-sequence learning framework for visual tracking, dubbed SeqTrack. It casts visual tracking as a sequence generation problem, which predicts object bounding boxes in an autoregressive fashion. This is different from prior Siamese trackers and transformer trackers, which rely on designing complicated head networks, such as classification and regression heads. SeqTrack only adopts a simple encoder-decoder transformer architecture. The encoder extracts visual features with a bidirectional transformer, while the decoder generates a sequence of bounding box values autoregressively with a causal transformer. The loss function is a plain cross-entropy. Such a sequence learning paradigm not only simplifies tracking framework, but also achieves competitive performance on benchmarks. For instance, SeqTrack gets 72.5% AUC on LaSOT, establishing a new…

Demographic Estimation from Face Images: Human vs. Machine Performance

Hu Han, Charles Otto, Xiaoming Liu +1 more

Computer Vision and Pattern Recognition · IEEE Transactions on Pattern Analysis and Machine Intelligence · 28 Oct 2014 · cited 330× · DOI ↗ · more like this ·

Demographic estimation entails automatic estimation of age, gender and race of a person from his face image, which has many potential applications ranging from forensics to social media. Automatic demographic estimation, particularly age estimation, remains a challenging problem because persons belonging to the same demographic group can be vastly different in their facial appearances due to intrinsic and extrinsic factors. In this paper, we present a generic framework for automatic demographic (age, gender and race) estimation. Given a face image, we first extract demographic informative features via a boosting algorithm, and then employ a hierarchical approach consisting of between-group classification, and within-group regression. Quality assessment is also developed to identify low-quality face images that are difficult to obtain reliable demographic estimates. Experimental results…

Heterogeneous Face Attribute Estimation: A Deep Multi-Task Learning Approach

Hu Han, Anil K. Jain, Fang Wang +2 more

cs.CV · June 2017 · cited 301× · arXiv ↗ · more like this ·

Face attribute estimation has many potential applications in video surveillance, face retrieval, and social media. While a number of methods have been proposed for face attribute estimation, most of them did not explicitly consider the attribute correlation and heterogeneity (e.g., ordinal vs. nominal and holistic vs. local) during feature representation learning. In this paper, we present a Deep Multi-Task Learning (DMTL) approach to jointly estimate multiple heterogeneous attributes from a single face image. In DMTL, we tackle attribute correlation and heterogeneity with convolutional neural networks (CNNs) consisting of shared feature learning for all the attributes, and category-specific feature learning for heterogeneous attributes. We also introduce an unconstrained face database (LFW+), an extension of public-domain LFW, with heterogeneous demographic attributes (age, gender, and…

Smooth Representation Clustering

Hu Han, Zhouchen Lin, Jianjiang Feng +1 more

Computer Vision and Pattern Recognition · June 2014 · cited 290× · DOI ↗ · more like this ·

Subspace clustering is a powerful technology for clustering data according to the underlying subspaces. Representation based methods are the most popular subspace clustering approach in recent years. In this paper, we analyze the grouping effect of representation based methods in depth. In particular, we introduce the enforced grouping effect conditions, which greatly facilitate the analysis of grouping effect. We further find that grouping effect is important for subspace clustering, which should be explicitly enforced in the data self-representation model, rather than implicitly implied by the model as in some prior work. Based on our analysis, we propose the SMooth Representation (SMR) model. We also propose a new affinity measure based on the grouping effect, which proves to be much more effective than the commonly used one. As a result, our SMR significantly outperforms the…

Age estimation from face images: Human vs. machine performance

Hu Han, Charles Otto, Anil K. Jain

Computer Vision and Pattern Recognition · June 2013 · cited 243× · DOI ↗ · more like this ·

There has been a growing interest in automatic age estimation from facial images due to a variety of potential applications in law enforcement, security control, and human-computer interaction. However, despite advances in automatic age estimation, it remains a challenging problem. This is because the face aging process is determined not only by intrinsic factors, e.g. genetic factors, but also by extrinsic factors, e.g. lifestyle, expression, and environment. As a result, different people with the same age can have quite different appearances due to different rates of facial aging. We propose a hierarchical approach for automatic age estimation, and provide an analysis of how aging influences individual facial components. Experimental results on the FG-NET, MORPH Album2, and PCSO databases show that eyes and nose are more informative than the other facial components in automatic age…

Mean-Variance Loss for Deep Age Estimation from a Face

Hongyu Pan, Hu Han, Shiguang Shan +1 more

Computer Vision and Pattern Recognition · June 2018 · cited 239× · DOI ↗ · more like this ·

Age estimation has wide applications in video surveillance, social networking, and human-computer interaction. Many of the published approaches simply treat age estimation as an exact age regression problem, and thus do not leverage a distribution's robustness in representing labels with ambiguity such as ages. In this paper, we propose a new loss function, called mean-variance loss, for robust age estimation via distribution learning. Specifically, the mean-variance loss consists of a mean loss, which penalizes difference between the mean of the estimated age distribution and the ground-truth age, and a variance loss, which penalizes the variance of the estimated age distribution to ensure a concentrated distribution. The proposed mean-variance loss and softmax loss are jointly embedded into Convolutional Neural Networks (CNNs) for age estimation. Experimental results on the FG-NET…

Unconstrained Face Recognition: Identifying a Person of Interest From a Media Collection

Lacey Best-Rowden, Hu Han, Charles Otto +2 more

Computer Vision and Pattern Recognition · IEEE Transactions on Information Forensics and Security · 19 Sep 2014 · cited 214× · DOI ↗ · more like this ·

As face recognition applications progress from constrained sensing and cooperative subjects scenarios (e.g., driver's license and passport photos) to unconstrained scenarios with uncooperative subjects (e.g., video surveillance), new challenges are encountered. These challenges are due to variations in ambient illumination, image resolution, background clutter, facial pose, expression, and occlusion. In forensic investigations where the goal is to identify a person of interest, often based on low quality face images and videos, we need to utilize whatever source of information is available about the person. This could include one or more video tracks, multiple still images captured by bystanders (using, for example, their mobile phones), 3-D face models constructed from image(s) and video(s), and verbal descriptions of the subject provided by witnesses. These verbal descriptions can be…

Cross-domain Face Presentation Attack Detection via Multi-domain Disentangled Representation Learning

Guoqing Wang, Hu Han, Shiguang Shan +1 more

cs.CV · 4 Apr 2020 · cited 212× · arXiv ↗ · more like this ·

Face presentation attack detection (PAD) has been an urgent problem to be solved in the face recognition systems. Conventional approaches usually assume the testing and training are within the same domain; as a result, they may not generalize well into unseen scenarios because the representations learned for PAD may overfit to the subjects in the training set. In light of this, we propose an efficient disentangled representation learning for cross-domain face PAD. Our approach consists of disentangled representation learning (DR-Net) and multi-domain learning (MD-Net). DR-Net learns a pair of encoders via generative models that can disentangle PAD informative features from subject discriminative features. The disentangled features from different domains are fed to MD-Net which learns domain-independent features for the final cross-domain face PAD task. Extensive experiments on several…

Distribution of the COVID-19 epidemic and correlation with population emigration from Wuhan, China

Ze-Liang Chen, Zhang Qi, Yi Lu +8 more

Modeling and Simulation · Chinese Medical Journal · 28 Feb 2020 · cited 204× · DOI ↗ · more like this ·

BACKGROUND: The ongoing new coronavirus pneumonia (Corona Virus Disease 2019, COVID-19) outbreak is spreading in China, but it has not yet reached its peak. Five million people emigrated from Wuhan before lockdown, potentially representing a source of virus infection. Determining case distribution and its correlation with population emigration from Wuhan in the early stage of the epidemic is of great importance for early warning and for the prevention of future outbreaks. METHODS: The official case report on the COVID-19 epidemic was collected as of January 30, 2020. Time and location information on COVID-19 cases was extracted and analyzed using ArcGIS and WinBUGS software. Data on population migration from Wuhan city and Hubei province were extracted from Baidu Qianxi, and their correlation with the number of cases was analyzed. RESULTS: The COVID-19 confirmed and death cases in Hubei…

Matching Composite Sketches to Face Photos: A Component-Based Approach

Hu Han, Brendan Klare, Kathryn Bonnen +1 more

Computer Vision and Pattern Recognition · IEEE Transactions on Information Forensics and Security · 21 Nov 2012 · cited 199× · DOI ↗ · more like this ·

The problem of automatically matching composite sketches to facial photographs is addressed in this paper. Previous research on sketch recognition focused on matching sketches drawn by professional artists who either looked directly at the subjects (viewed sketches) or used a verbal description of the subject's appearance as provided by an eyewitness (forensic sketches). Unlike sketches hand drawn by artists, composite sketches are synthesized using one of the several facial composite software systems available to law enforcement agencies. We propose a component-based representation (CBR) approach to measure the similarity between a composite sketch and mugshot photograph. Specifically, we first automatically detect facial landmarks in composite sketches and face photos using an active shape model (ASM). Features are then extracted for each facial component using multiscale local binary…

Unsupervised Adversarial Domain Adaptation for Cross-Domain Face Presentation Attack Detection

Guoqing Wang, Hu Han, Shiguang Shan +1 more

Computer Vision and Pattern Recognition · IEEE Transactions on Information Forensics and Security · 15 Jun 2020 · cited 151× · DOI ↗ · more like this ·

Face presentation attack detection (PAD) is essential for securing the widely used face recognition systems. Most of the existing PAD methods do not generalize well to unseen scenarios because labeled training data of the new domain is usually not available. In light of this, we propose an unsupervised domain adaptation with disentangled representation (DR-UDA) approach to improve the generalization capability of PAD into new scenarios. DR-UDA consists of three modules, i.e., ML-Net, UDA-Net and DR-Net. ML-Net aims to learn a discriminative feature representation using the labeled source domain face images via metric learning. UDA-Net performs unsupervised adversarial domain adaptation in order to optimize the source domain and target domain encoders jointly, and obtain a common feature space shared by both domains. As a result, the source domain PAD model can be effectively transferred…

Local Relationship Learning With Person-Specific Shape Regularization for Facial Action Unit Detection

Xuesong Niu, Hu Han, Songfan Yang +2 more

Computer Vision and Pattern Recognition · IEEE Conference Proceedings · June 2019 · cited 141× · DOI ↗ · more like this ·

Encoding individual facial expressions via action units (AUs) coded by the Facial Action Coding System (FACS) has been found to be an effective approach in resolving the ambiguity issue among different expressions. While a number of methods have been proposed for AU detection, robust AU detection in the wild remains a challenging problem because of the diverse baseline AU intensities across individual subjects, and the weakness of appearance signal of AUs. To resolve these issues, in this work, we propose a novel AU detection method by utilizing local information and the relationship of individual local face regions. Through such a local relationship learning, we expect to utilize rich local information to improve the AU detection robustness against the potential perceptual inconsistency of individual local regions. In addition, considering the diversity in the baseline AU intensities…

AgeNet: Deeply Learned Regressor and Classifier for Robust Apparent Age Estimation

Xin Liu, Shaoxin Li, Meina Kan +6 more

Computer Vision and Pattern Recognition · December 2015 · cited 141× · DOI ↗ · more like this ·

Apparent age estimation from face image has attracted more and more attentions as it is favorable in some real-world applications. In this work, we propose an end-to-end learning approach for robust apparent age estimation, named by us AgeNet. Specifically, we address the apparent age estimation problem by fusing two kinds of models, i.e., real-value based regression models and Gaussian label distribution based classification models. For both kind of models, large-scale deep convolutional neural network is adopted to learn informative age representations. Another key feature of the proposed AgeNet is that, to avoid the problem of over-fitting on small apparent age training set, we exploit a general-to-specific transfer learning scheme. Technically, the AgeNet is first pre-trained on a large-scale web-collected face dataset with identity label, and then it is fine-tuned on a large-scale…

Continuous authentication of mobile user: Fusion of face image and inertial Measurement Unit data

David Crouse, Hu Han, Deepak Chandra +2 more

Signal Processing · May 2015 · cited 109× · DOI ↗ · more like this ·

Mobile devices can carry large amounts of personal data, but are often left unsecured. PIN locks are inconvenient to use and thus have seen low adoption (33% of users). While biometrics are beginning to be used for mobile device authentication, they are used only for initial unlock. Mobile devices secured with only login authentication are still vulnerable to data theft when in an unlocked state. This paper introduces our work on a face-based continuous authentication system that operates in an unobtrusive manner. We present a methodology for fusing mobile device (unconstrained) face capture with gyroscope, accelerometer, and magnetometer data to correct for camera orientation and, by extension, the orientation of the face image. Experiments demonstrate (i) improvement of face recognition accuracy from face orientation correction, and (ii) efficacy of the prototype continuous…

DISC: Learning from Noisy Labels via Dynamic Instance-Specific Selection and Correction

Yifan Li, Hu Han, Shiguang Shan +1 more

Artificial Intelligence · June 2023 · cited 103× · DOI ↗ · more like this ·

Existing studies indicate that deep neural networks (DNNs) can eventually memorize the label noise. We observe that the memorization strength of DNNs towards each instance is different and can be represented by the confidence value, which becomes larger and larger during the training process. Based on this, we propose a Dynamic Instance-specific Selection and Correction method (DISC) for learning from noisy labels (LNL). We first use a two- view-based backbone for image classification, obtaining confidence for each image from two views. Then we propose a dynamic threshold strategy for each instance, based on the momentum of each instance's memorization strength in previous epochs to select and correct noisy labeled data. Benefiting from the dynamic threshold strategy and two-view learning, we can effectively group each instance into one of the three subsets (i.e., clean, hard, and…

Live face video vs. spoof face video: Use of moir&#x00E9; patterns to detect replay video attacks

Keyurkumar J. Patel, Hu Han, Anil K. Jain +1 more

Signal Processing · May 2015 · cited 101× · DOI ↗ · more like this ·

With the wide deployment of face recognition systems in applications from border control to mobile device unlocking, the combat of face spoofing attacks requires increased attention; such attacks can be easily launched via printed photos, video replays and 3D masks. We address the problem of facial spoofing detection against replay attacks based on the analysis of aliasing in spoof face videos. The application domain of interest is mobile phone unlock. We analyze the moiré pattern aliasing that commonly appears during the recapture of video or photo replays on a screen in different channels (R, G, B and grayscale) and regions (the whole frame, detected face, and facial component between the nose and chin). Multi-scale LBP and DSIFT features are used to represent the characteristics of moiré patterns that differentiate a replayed spoof face from a live face (face present). Experimental…

Improving Cross-database Face Presentation Attack Detection via Adversarial Domain Adaptation

Guoqing Wang, Hu Han, Shiguang Shan +1 more

Computer Vision and Pattern Recognition · June 2019 · cited 94× · DOI ↗ · more like this ·

Face recognition (FR) is being widely used in many applications from access control to smartphone unlock. As a result, face presentation attack detection (PAD) has drawn increasing attentions to secure the FR systems. Traditional approaches for PAD mainly assume that training and testing scenarios are similar in imaging conditions (illumination, scene, camera sensor, etc.), and thus may lack good generalization capability into new application scenarios. In this work, we propose an end-to-end learning approach to improve PAD generalization capability by utilizing prior knowledge from source domain via adversarial domain adaptation. We first build a source domain PAD model optimized with triplet loss. Subsequently, we perform adversarial domain adaptation w.r.t. the target domain to learn a shared embedding space by both the source and target domain models, in which the discriminator…

The FaceSketchID System: Matching Facial Composites to Mugshots

Scott Klum, Hu Han, Brendan Klare +1 more

Computer Vision and Pattern Recognition · IEEE Transactions on Information Forensics and Security · 29 Sep 2014 · cited 91× · DOI ↗ · more like this ·

Facial composites are widely used by law enforcement agencies to assist in the identification and apprehension of suspects involved in criminal activities. These composites, generated from witness descriptions, are posted in public places and media with the hope that some viewers will provide tips about the identity of the suspect. This method of identifying suspects is slow, tedious, and may not lead to the timely apprehension of a suspect. Hence, there is a need for a method that can automatically and efficiently match facial composites to large police mugshot databases. Because of this requirement, facial composite recognition is an important topic for biometrics researchers. While substantial progress has been made in nonforensic facial composite (or viewed composite) recognition over the past decade, very little work has been done using operational composites relevant to law…

Sketch based face recognition: Forensic vs. composite sketches

Scott Klum, Hu Han, Anil K. Jain +1 more

Computer Vision and Pattern Recognition · June 2013 · cited 77× · DOI ↗ · more like this ·

Facial sketches are widely used by law enforcement agencies to assist in the identification and apprehension of suspects involved in criminal activities. Sketches used in forensic investigations are either drawn by forensic artists (forensic sketches) or created with computer software (composite sketches) following the verbal description provided by an eyewitness or the victim. These sketches are posted in public places and in media in hopes that some viewers will provide tips about the identity of the suspect. This method of identifying suspects is slow and tedious and may not lead to apprehension of the suspect. Hence, there is a need for a method that can automatically and quickly match facial sketches to large police mugshot databases. We address the problem of automatic facial sketch to mugshot matching and, for the first time, compare the effectiveness of forensic sketches and…

Automatic Engagement Prediction with GAP Feature

Xuesong Niu, Hu Han, Jiabei Zeng +5 more

Human-Computer Interaction · 2 Oct 2018 · cited 67× · DOI ↗ · more like this ·

In this paper, we propose an automatic engagement prediction method for the Engagement in the Wild sub-challenge of EmotiW 2018. We first design a novel Gaze-AU-Pose (GAP) feature taking into account the information of gaze, action units and head pose of a subject. The GAP feature is then used for the subsequent engagement level prediction. To efficiently predict the engagement level for a long-time video, we divide the long-time video into multiple overlapped video clips and extract GAP feature for each clip. A deep model consisting of a Gated Recurrent Unit (GRU) layer and a fully connected layer is used as the engagement predictor. Finally, a mean pooling layer is applied to the per-clip estimation to get the final engagement level of the whole video. Experimental results on the validation set and test set show the effectiveness of the proposed approach. In particular, our approach…

Improving 2D Face Recognition via Discriminative Face Depth Estimation

Jiyun Cui, Hao Zhang, Hu Han +2 more

Computer Vision and Pattern Recognition · February 2018 · cited 65× · DOI ↗ · more like this ·

As face recognition progresses from constrained scenarios to unconstrained scenarios, new challenges such as large pose, bad illumination, and partial occlusion, are encountered. While 3D or multi-modality RGB-D sensors are helpful for face recognition systems to achieve robustness against these challenges, the requirement of new sensors limits their application scenarios. In our paper, we propose a discriminative face depth estimation approach to improve 2D face recognition accuracies under unconstrained scenarios. Our discriminative depth estimation method uses a cascaded FCN and CNN architecture, in which FCN aims at recovering the depth from an RGB image, and CNN retains the separability of individual subjects. The estimated depth information is then used as a complementary modality to RGB for face recognition tasks. Experiments on two public datasets and a dataset we collect show…

Multi-Modal Face Presentation Attack Detection via Spatial and Channel Attentions

Guoqing Wang, Chuanxin Lan, Hu Han +2 more

Signal Processing · June 2019 · cited 52× · DOI ↗ · more like this ·

Face presentation attack detection (PAD) has drawn increasing attentions to secure face recognition (FR) systems which are being widely used in many applications from access control to smartphone unlock. Traditional approaches for PAD may lack good generalization capability into new application scenarios due to the limited number of subjects and data modality. In this work, we propose an end-to-end multi-modal fusion approach via spatial and channel attention to improve PAD performance on CASIA-SURF. Specifically, We first build four branches integrated with spatial and channel attention module to obtain the uniform features of different modalities, i.e., RGB, Depth, IR and the fused modality with 9 channels which concatenating three modalities. Subsequently, the features extracted from the four branches are concatenated and fed into the shared layers to learn more discriminative…