Author

Lejun Yu

32 papers · 2005–2024

2005peak 5 in 20162024

Frequent co-authors

Papers

showing 1–25 of 32 RSS

Motion Adaptive Deinterlacing with Accurate Motion Detection and Anti-Aliasing Interpolation Filter

Lejun Yu, Jintao Li, Yongdong Zhang +1 more

Computer Vision and Pattern Recognition · IEEE Transactions on Consumer Electronics · May 2006 · cited 24× · DOI ↗ · more like this ·

Both the motion-detection and infra-field interpolation filter are important factors affect the efficiency of motion adaptive de-interlacing. New accurate motion detection (AMD) algorithm is proposed to improve the accuracy of motion detection, which reduces the possibility of error motion detection with a median filter. To improve the efficiency of intra-field interpolation deinterlacing in moving regions, an anti-aliasing interpolation filter (AAIF) is proposed, which is better than the typical windowed sine function. The simulation results show that peak signal noise ratio (PSNR) of our proposed deinterlacing method is 0.5-7.5 dB higher than that of previous studies and attains the best quality of subjective view.

BNU-LSVED: a multimodal spontaneous expression database in educational environment

Bo Sun, Qinglan Wei, Jun He +2 more

Artificial Intelligence · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 14 Sep 2016 · cited 13× · DOI ↗ · more like this ·

In the field of pedagogy or educational psychology, emotions are treated as very important factors, which are closely associated with cognitive processes. Hence, it is meaningful for teachers to analyze students’ emotions in classrooms, thus adjusting their teaching activities and improving students ’ individual development. To provide a benchmark for different expression recognition algorithms, a large collection of training and test data in classroom environment has become an acute problem that needs to be resolved. In this paper, we present a multimodal spontaneous database in real learning environment. To collect the data, students watched seven kinds of teaching videos and were simultaneously filmed by a camera. Trained coders made one of the five learning expression labels for each image sequence extracted from the captured videos. This subset consists of 554 multimodal…

Single-image super-resolution based on regularization with stationary gradient fidelity

Lejun Yu, Siming Cao, Jun He +2 more

Computer Vision and Pattern Recognition · October 2017 · cited 11× · DOI ↗ · more like this ·

Single-image super-resolution, reconstructing a high resolution (HR) image from a low resolution (LR) one, is an ill-posed problem. Regularization, a method for ill-posed problem, is widely used in super-resolution. The definition of the fidelity term in regularization energy is a key to the performance of regularization based super-resolution. Traditional fidelity is defined as the energy of error image between down-sampled version of HR image estimated and the observed LR image. Since the mpixels not in sampled positions are not considered, the gradient of traditional fidelity is non-stationary. To avoid this problem in traditional fidelity, we propose to define the fidelity term as the energy of error image between the estimated HR image estimate and its blurred version. When gradient decent is used to optimize the regularization energy, we propose an error interpolation fidelity…

Multimodal Facial Expression Recognition Based on Dempster-Shafer Theory Fusion Strategy

Qihua Xu, Bo Sun, Jun He +3 more

Computer Vision and Pattern Recognition · May 2018 · cited 10× · DOI ↗ · more like this ·

In this paper, we present our facial expression recognition framework for the Multimodal Emotion Recognition Challenge 2017. The task of the challenge is to recognize one of eight facial emotions in short video clips extracted from Chinese films, TV plays and talk shows. Considering the fact that each video is given only one video-level label, while each frame of the video is unlabeled, we improved the unsupervised auto-coder and used it for feature extraction. In addition, we have extracted other features, including traditional hand-craft features and deep learning features. Finally, the random forest algorithm was used to classify different emotional labels, and followed by a decision level fusion method based on Dempster-Shafer evidence theory. The macro average precision(MAP) of our best submission result achieves 59.68% on the testing set, which significantly outperforms the…

Research on video capture scheme and face recognition algorithms in a class attendance system

Jun He, Yijia Zhao, Bo Sun +1 more

Computer Vision and Pattern Recognition · 6 Sep 2017 · cited 5× · DOI ↗ · more like this ·

Intelligent classroom is a trending topic and many sophisticated technologies are now used in classrooms. Class attendance is an important part of teaching management, meanwhile video monitoring techniques and face recognition algorithms have also made great progress. However, few researches have focused on the applications of the latter in actual classrooms. Aiming at an automatic class attendance system, the paper proposes a classroom video capture scheme, whose objective is to obtain the monitoring videos, meeting the requirements of the face recognition algorithms. Also the paper analyzes various factors' impacts of the face recognition, then find out the face recognition algorithms which perform better in our experiments. Finally, the algorithms would get the class attendance according to the video files obtained by improved monitoring system. After the field test, the proposed…

Action recognition based on multiple key motion history images

Dongxue Li, Lejun Yu, Jun He +2 more

Computer Vision and Pattern Recognition · November 2016 · cited 5× · DOI ↗ · more like this ·

Human action recognition in video sequences is an important research topic in computer vision, and motion history image (MHI) is widely taken for recognition due to its simplicity. However, it may be not robust to describe an action by only a single MHI. Therefore, an action recognition scheme by using multiple key MHIs (MKMHIs) is proposed. Firstly, an adaptive method for key MHIs selection is proposed based on entropy of MHIs. Then a new combined feature vector of MHIs by spatial pyramid matching (SPM) is defined to describe spatiotemporal characteristics of actions. Here, the proposed solution is composed of SPM two dimensional entropy (2D-entropy) of MHI and SPM Zernike moment of motion history image edge (MHIE), and our combined feature is lower compared with Local Binary Pattern Histogram(LBP_H). At last, action recognition is performed by SVM and voting. Experimental results show…

Format-Independent Motion Content Description based on Spatiotemporal Visual Sensitivity

Xuefeng Pan, Jintao Li, Yongdong Zhang +2 more

Computer Vision and Pattern Recognition · IEEE Transactions on Consumer Electronics · May 2007 · cited 5× · DOI ↗ · more like this ·

With the extensive application of digital video technology, developing format-independent motion describing method is of great significance for retrieving and searching video content in different formats. In this paper, a format-independent motion describing method for video content is proposed. The features based on visual sensitivity are extracted from spatiotemporal slice. Since the same video content gives the same visual stimuli to visual perception, the method based on this kind of visual sensitivity related feature is format-independent. The experiments show the feature proposed is sensitive to the variation of video content and robust to the variation of video format. The motion describing method proposed is format-independent.

Feedback evaluations to promote image captioning

Jun He, Yijia Zhao, Bo Sun +1 more

Computer Vision and Pattern Recognition · IET Image Processing · 4 Jun 2020 · cited 4× · DOI ↗ · more like this ·

Image captioning can be treated as a policy gradient problem. A retrieval model to obtain the discriminability score to distinguish between two images, given the caption for one of them, has been proposed previously; the discriminability score and one of the image captioning evaluation metrics were optimised using policy gradient. Based on this, two methods to evaluate the caption and caption‐generating process, referred to as feedback evaluations, are proposed in this study. The results of the evaluations were used to improve the model. First, an auxiliary retrieval loss (ARL) is introduced to evaluate the generated caption to improve the discriminability of the model. ARL has been utilised as a feedback evaluation method because it calculates similarity between the generated caption and convolutional neural network features. With ARL, a higher similarity and better discriminability…

Facial Emotion and Action Unit Recognition based on Bayesian Network

Jun He, Xiaocui Yu, Lejun Yu +1 more

Computer Vision and Pattern Recognition · ACM Proceedings · 23 Oct 2019 · cited 4× · DOI ↗ · more like this ·

Understanding human facial expressions is one of the key steps to achieving human-computer interaction. In this paper, we report on advances we have made in building a Bayesian network (BN) for facial expressions recognition based on the CK+ datasets. One of the motivating factors for using a BN is that it can simulate the process of generation and cognition of facial expressions and action units (AUs) realistically. The constructed BN comprises five layers; besides commonly-used expressions, AUs, and feature layers, brain cognition and facial muscles layers are firstly proposed and innovatively designed for the recognition of facial expressions. Further, via deep analysis, we firstly select the most proper facial landmark points, and then fuse their displacement feature and texture features as low-level features. The results of experiments indicate that the two layer we proposed and…

Multi-feature-based robust face detection and coarse alignment method via multiple kernel learning

Bo Sun, Di Zhang, Jun He +2 more

Computer Vision and Pattern Recognition · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 21 Oct 2015 · cited 4× · DOI ↗ · more like this ·

Face detection and alignment are two crucial tasks to face recognition which is a hot topic in the field of defense and security, whatever for the safety of social public, personal property as well as information and communication security. Common approaches toward the treatment of these tasks in recent years are often of three types: template matching-based, knowledge-based and machine learning-based, which are always separate-step, high computation cost or fragile robust. After deep analysis on a great deal of Chinese face images without hats, we propose a novel face detection and coarse alignment method, which is inspired by those three types of methods. It is multi-feature fusion with Simple Multiple Kernel Learning<sup>1</sup> (Simple-MKL) algorithm. The proposed method is contrasted with competitive and related algorithms, and demonstrated to achieve promising results.

Fast Frame/Field Coding for H.264/AVC

Lejun Yu, Jintao Li, Yanfei Shen

Signal Processing · 22 Sep 2006 · cited 4× · DOI ↗ · more like this ·

Picture-adaptive frame/field (PAFF) coding and macroblock-adaptive frame/field (MBAFF) coding in the new H.264/AVC can improve compression efficiency for interlaced video coding. It is assumed that the optimal solution can be found by employing a multi-pass strategy, which encodes a macroblock or picture with all possible modes and chooses the one with smallest rate-distortion cost, such as the threepass coding for PAFF and MBAFF coding in reference of H.264/AVC. However, this coding strategy needs high computational complexity. In this paper, a novel single-pass algorithm based on motion activity detection is proposed to predict the mode of PAFF and MBAFF coding in an analysis stage. The simulation results show that the proposed scheme can reduce computational complexity by approximate 60%-75% compared to the three-pass coding method, while maintaining grate coding efficiency.

NIR-VIS Heterogeneous Face Synthesis via Enhanced Asymmetric CycleGAN

Yinghui Zhang, Chun Liu, Bo Sun +2 more

Computer Vision and Pattern Recognition · 18 Jul 2021 · cited 3× · DOI ↗ · more like this ·

Heterogeneous face image involves many fields, among which the near infrared (NIR) and visible (VIS) face image recognition is a hot field. NIR image has excellent imaging effect under extreme illumination conditions. Although few NIR images are available, unpaired image-to-image translation provides a solution for converting NIR-VIS to each other. However, NIR image contains relatively poor information, while VIS image contains relatively rich information, which results in asymmetry between NIR and VIS, and not suitable for unpaired translation based on two domains of the same complexity. In this paper, an enhanced asymmetric CycleGAN(EN-ASGAN) with edge retention module, auxiliary encoder module and generators equipped with different sizes is used to convert NIR-VIS face images. To validate the effective of EN-ASGAN, we conduct experiments on CASIA NIR-VIS 2.0 dataset. The…

BNU-LCSAD: A video database for classroom student action recognition

Bo Sun, Kaijie Zhao, Yongkang Xiao +4 more

Computer Vision and Pattern Recognition · 18 Nov 2019 · cited 3× · DOI ↗ · more like this ·

With the development and application of digital cameras, especially in education, a great number of digital video recordings are produced in classrooms. Taking Beijing Normal University as an example, 3.4 TB of videos are recorded every day in more than 200 classrooms. Such huge data is beneficial for us, computer vision researchers, to automatically recognize students' classroom actions and even evaluate the quality of classroom teaching. To focus action recognition on students, we propose Beijing Normal University Large-scale Classroom Student Action Database version 1.0(BNU-LCSAD) which is the first large-scale classroom student action database for student action recognition and consists of 10 classroom student action classes from digital camera recordings at BNU. We introduce the construct and label Processing of this database in detail. In Addition , we provide baseline of student…

Mode activity based adaptive fast intra mode decision for HEVC/H.265

Lejun Yu, Feng‐Xiang Ge, Jun He +2 more

Signal Processing · October 2015 · cited 3× · DOI ↗ · more like this ·

In the intra coding of HEVC/H.265, the computational complexity of intra mode decision is very high because up to 35 intra prediction modes are supported. Though a fast intra mode decision based on rough mode decision (RMD) is adopted in reference software HM12.0, only a fixed number of intra modes are selected for rate distortion optimization (RDO). In this paper, an adaptive fast intra mode decision for HEVC/H.265 is proposed. We define mode activity by analyzing the rough costs of all modes, and the computational complexity of mode activity is very low. Based on the analysis of relation between mode activity and rate distortion cost gain, we propose to adaptively reduce the number of candidate intra modes for RDO by utilizing the mode activity, so accelerated intra coding is realized. Compared with the RMD in reference code HM12.0 of HEVC/H.265, the proposed algorithm can reduce…

Automatically Predict Question Difficulty for Reading Comprehension Exercises

Jun He, Peng Li, Bo Sun +2 more

Artificial Intelligence · 2021 IEEE 33rd International Conference on Tools with Artificial Intelligence (ICTAI) · November 2021 · cited 2× · DOI ↗ · more like this ·

Question difficulty is a critical indicator for educational examination and personalized learning resources recommendation. Its evaluation mainly based on experts’ experience, which is both subjective and labor intensive. Recently, many studies pay more attention to using neural network for question difficulty prediction (QDP). Though these methods have improved efficiency of difficulty prediction, they only regarded the difficulty prediction task as a simple classification or prediction task, which ignored the influence of the input text relation on it, such as confusion relation of multiple choice for English reading comprehension items. Therefore, in this paper, we proposed a Convolutional Neural Network with Multi-view attention (MACNN) to extract different relation from multiple-parts text in reading comprehension exercises with multiple-choice for automatically predicting question…

Complexity scalable motion estimation based on modes pre-selection in H.264

Dougmmg Zhang, Chao Huang, Shouxun Lin +2 more

Signal Processing · 28 Jun 2005 · cited 2× · DOI ↗ · more like this ·

H.264 allows tree-structured partitioning motion estimation performing on multiple reference frames. This new feature improves the prediction accuracy of inter-coding blocks significantly, but it is extremely computational intensive. The complexity of motion estimation increases linearly with the number of used reference frames and is in proportion to the number of modes. This disables some applications on computations-constrained platform. Complexity scalable algorithm is an algorithm that has several computation scales, each of which adapts to certain computation power platform. In this paper, we propose a complexity scalable algorithm for motion estimation in H.264 based on mode preselection in multiple reference frames. This proposed algorithm has 4 computation scales consisting of 72%, 47%, 33% and 20%. Simulation results show that this algorithm can effectively reduce complexity…

Analyses and Benchmark of a Spontaneous Student Affect Database

Bo Sun, Sixu Lu, Yang Wen +2 more

Computer Science Applications · November 2021 · cited 1× · DOI ↗ · more like this ·

BNU-LSVED2.0 is the first large-scale spontaneous and multi-modal student affect database adapted to the classroom environment. On the basis of BNU-LSVED2.0, BNU-SDED is created. This study analyzes BNU-LSVED2.0 and BNU-SDED and develops a benchmark for BNU-SDED, which can provide some insights for future research on education improvement and emotion recognition. Firstly, student personality traits are analyzed to ensure the validity of data. Secondly, the relationship between PAD emotional dimensions and students’ emotional states in learning is analyzed. Based on these analyses, a set of mapping labels between categorical emotions and PAD dimensions is presented. Thirdly, BNU-SDED is used to perform basic baseline experiments. The results demonstrate the effectiveness of the data and reveal the trends behind students’ affects in learning. These analyses can benefit both developers and…

Bipolar Disorder Classification based on Multimodal Recordings

Siming Cao, Huanqing Yan, Penghao Rao +5 more

Signal Processing · 15 Oct 2021 · cited 1× · DOI ↗ · more like this ·

Automatic bipolar disorder classification is a challenging task. In this paper, we mainly focus on BD classification from acoustic, visual, and textual modalities. We highlight three aspects of our methods: 1) besides the baseline features, we explore and fuse some hand-crafted and deep learned features from all available modalities including acoustic, visual, and textual modalities. It should be noted that we extracted the textual modality by using the voice translation tool according to the acoustic modality; 2) Considering the fact that each video is given only one video-level label, while each frame of the video is unlabeled, we use the unsupervised Convolutional Auto-Encoder (CAE) and used it for feature extraction. 3) Due to the dataset is too small to train Convolutional Neural Network (CNN), so we decide to pre-train the CNN on other emotion datasets. The experimental results…

Regularized single-image super-resolution based on progressive gradient estimation

Lejun Yu, Xiaoyu Wu, Feng‐Xiang Ge +3 more

Computer Vision and Pattern Recognition · September 2015 · cited 1× · DOI ↗ · more like this ·

Gradient domain optimization is widely used in regularized image super-resolution, in which the gradient of high resolution (HR) is estimated for calculating the regularization energy. In this paper, a progressive gradient estimation (PGE) is proposed. In PGE, the gradient of the reconstructed HR image in the previous round of optimization is taken as the estimated gradient in the current round. Then, the estimated image gradient is progressively improved. When the estimated image gradient converges, a high quality HR image can be reconstructed. Experimental results show that the reconstructed HR images by PGE have good qualitative and quantitative performances.

Sparse presentation based classification with position-weighted block dictionary

Jun He, Tian Zuo, Bo Sun +4 more

Computer Vision and Pattern Recognition · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 25 Feb 2014 · cited 1× · DOI ↗ · more like this ·

This paper is aiming at applying sparse representation based classification (SRC) on general objects of a certain scale. Authors analyze the characteristics of general object recognition and propose a position-weighted block dictionary (PWBD) based on sparse presentation and design a framework of SRC with it (PWBD-SRC). Principle and implementation of PWBD-SRC have been introduced in the article, and experiments on car models have been given in the article. From experimental results, it can be seen that with position-weighted block dictionary (PWBD) not only the dictionary scale can be effectively reduced, but also roles of image blocks taking in representing a whole image can be embodied to a certain extent. In reorganization application, an image only containing partial objects can be identified with PWBD-SRC. Besides, rotation and perspective robustness can be achieved. Finally, a…

A novel method based on lines clustering for extracting multi-ROI in images with simple background

Jun He, Bo Sun, Lejun Yu

Computer Vision and Pattern Recognition · August 2011 · cited 1× · DOI ↗ · more like this ·

Since application of target recognition is mostly used in multi-object images, extracting interested objects efficiently is much necessary. Taking account of non-curve but regular-shaped objects on simple background, the authors have proposed a novel method based on lines clustering for extracting multi-ROI. Steps of the method have been demonstrated in the article. Experimental data have also given and proved the method effective and practical.

Fast Picture and Macroblock Level Adaptive Frame/Field Coding for H.264

Lejun Yu, Jintao Li, Yongdong Zhang

Signal Processing · December 2006 · cited 1× · DOI ↗ · more like this ·

Picture adaptive frame/field (PAFF) coding and macroblock adaptive frame/field (MBAFF) coding in the new video coding standard H.264 can improve compression efficiency for interlaced video coding. It is assumed that the optimal solution can be found by employing a multi-pass strategy, which encodes a macroblock (MB) or picture with all possible modes and chooses the one with smallest rate-distortion cost. However, this coding strategy needs high computational complexity. In this paper, a novel single-pass algorithm based on motion activity detection is proposed to predict the mode of PAFF and MBAFF coding in an analysis stage. The simulation results show that the proposed scheme can reduce computational complexity by approximate 60%~75% compared to the 3-pass encoder of reference software, while maintaining grate coding efficiency

Zero-shot human-object interaction detection via learning from the data itself

Bo Sun, Dongliang Zhao, Sixu Lu +3 more

Computer Vision and Pattern Recognition · 22 Oct 2024 · cited 0× · DOI ↗ · more like this ·

Human-Object Interaction (HOI) Detection aims to locate and recognize HOI instances in the form of in images or videos. However, the significant cost of manpower and resources for annotations poses challenges, particularly in addressing the long-tail or even zero-shot distribution problem. Additionally, the generation of visual features from fixed semantic information encounters the issue of a lack of diversity. In response of these challenges, we develop a novel Visual-Semantic and Multi-source Feature Generation (VSMG) network for zero-shot human-object interaction detection. Firstly, by combining the visual-semantic GAN, the model not only generates visual/semantic features from corresponding semantic/visual ones but also introduces the diverse unseen visual features during the training phase. Secondly, by employing a knowledge-aware relation graph, the model encodes the relations…

BNU-CMHE: A Chinese Multi-Modal Heuristic Education Dataset from Beijing Normal University

Bo Sun, Xin Xu, Hang Li +2 more

Computer Science Applications · 19 Apr 2024 · cited 0× · DOI ↗ · more like this ·

Heuristic teaching is an active teaching pattern which emphasizes elicitation. Corresponding high-quality sam-ples are primal for developing Artificial Intelligence technology for evaluating heuristic. Thus, we present BNU -CMHE, i.e., a Chinese Multi-modal Heuristic Education Dataset from Beijing Normal University. It includes 3 tasks, namely heuristic teaching action detection, recognitionand classification. It contains 16 videos with audio, about 756 minutes in total, recorded from real lectures. 671 teaching action clips are highlighted. A heuristic level is labeled to each video. Time span, action category, heuristic level and 11 education characteristic labels are given to each action clip. Different from other multi-modal action-related datasets, the textual information is much more important than videos and audios. The dataset not only introduces new challenges comparing with…

Weakly Supervised Object Detection with Position Information of Convolution Neural Network

Bo Sun, Huanqing Yan, Jun He +2 more

Computer Vision and Pattern Recognition · 2022 IEEE 10th International Conference on Information, Communication and Networks (ICICN) · 23 Aug 2022 · cited 0× · DOI ↗ · more like this ·

Weakly Supervised Object Detection (WSOD) is a task that training object detector with only image-level label which typically means the category label. In this task, it is difficult to train a separate position regressor due to the lack of bounding box label. To solve this problem, some methods based on Multiple Instance Learning which establishes an instance (box) classifier by learning information from bag (image) are proposed by transforming the detection task into classification one. However, classifiers tend to find the most discriminative local region of an object, which leads to the problem of local optimization. In this paper, we utilize a regularization restriction method to obtain more global attention information by using the multi-layer features of convolutional neural network. Further, we use the feature enhanced by attention to locate more complete object. Meanwhile, the…