arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

XuGang

228 papers · 1997–2026

1997: 11998: 01999: 02000: 32001: 12002: 52003: 42004: 22005: 02006: 52007: 62008: 102009: 72010: 102011: 92012: 92013: 32014: 112015: 122016: 112017: 102018: 72019: 52020: 112021: 122022: 112023: 162024: 202025: 172026: 10
1997peak 20 in 20242026

Frequent co-authors

Papers

showing 1–25 of 228 RSS

Speech enhancement based on deep denoising autoencoder

Xugang Lu, Yu Tsao, Shigeki Matsuda +1 more

Signal Processing · · cited 870× · DOI ↗ · more like this ·

We previously have applied deep autoencoder (DAE) for noise reduction and speech enhancement. However, the DAE was trained using only clean speech. In this study, we further introduce an explicit denoising process in learning the DAE. In training the DAE, we still adopt greedy layer-wised pretraining plus fine tuning strategy. In pretraining, each layer is trained as a one hidden layer neural autoencoder (AE) using noisy-clean speech pairs as input and output (or transformed noisy-clean speech pairs by preceding AEs). Fine tuning was done by stacking all AEs with pretrained parameters for initialization. The trained DAE is used as a filter for speech estimation when noisy speech is given. Speech enhancement experiments were done to examine the performance of the trained denoising DAE. Noise reduction, speech distortion, and perceptual evaluation of speech quality (PESQ) criteria are…

MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement

Szu-Wei Fu, Cheng Yu, Tsun-An Hsieh +4 more

cs.SD · · cited 327× · arXiv ↗ · more like this ·

The discrepancy between the cost function used for training a speech enhancement model and human auditory perception usually makes the quality of enhanced speech unsatisfactory. Objective evaluation metrics which consider human perception can hence serve as a bridge to reduce the gap. Our previously proposed MetricGAN was designed to optimize objective metrics by connecting the metric with a discriminator. Because only the scores of the target evaluation functions are needed during training, the metrics can even be non-differentiable. In this study, we propose a MetricGAN+ in which three training techniques incorporating domain-knowledge of speech processing are proposed. With these techniques, experimental results on the VoiceBank-DEMAND dataset show that MetricGAN+ can increase PESQ score by 0.3 compared to the previous MetricGAN and achieve state-of-the-art results (PESQ score =…

End-to-End Waveform Utterance Enhancement for Direct Evaluation Metrics Optimization by Fully Convolutional Neural Networks

Szu-Wei Fu, Tao-Wei Wang, Yu Tsao +2 more

stat.ML · · cited 300× · arXiv ↗ · more like this ·

Speech enhancement model is used to map a noisy speech to a clean speech. In the training stage, an objective function is often adopted to optimize the model parameters. However, in most studies, there is an inconsistency between the model optimization criterion and the evaluation criterion on the enhanced speech. For example, in measuring speech intelligibility, most of the evaluation metric is based on a short-time objective intelligibility (STOI) measure, while the frame based minimum mean square error (MMSE) between estimated and clean speech is widely used in optimizing the model. Due to the inconsistency, there is no guarantee that the trained model can provide optimal performance in applications. In this study, we propose an end-to-end utterance-based speech enhancement framework using fully convolutional neural networks (FCN) to reduce the gap between the model optimization and…

Raw Waveform-based Speech Enhancement by Fully Convolutional Networks

Szu-Wei Fu, Yu Tsao, Xugang Lu +1 more

stat.ML · · cited 213× · arXiv ↗ · more like this ·

This study proposes a fully convolutional network (FCN) model for raw waveform-based speech enhancement. The proposed system performs speech enhancement in an end-to-end (i.e., waveform-in and waveform-out) manner, which dif-fers from most existing denoising methods that process the magnitude spectrum (e.g., log power spectrum (LPS)) only. Because the fully connected layers, which are involved in deep neural networks (DNN) and convolutional neural networks (CNN), may not accurately characterize the local information of speech signals, particularly with high frequency components, we employed fully convolutional layers to model the waveform. More specifically, FCN consists of only convolutional layers and thus the local temporal structures of speech signals can be efficiently and effectively preserved with relatively few weights. Experimental results show that DNN- and CNN-based models…

Complex spectrogram enhancement by convolutional neural network with multi-metrics learning

Szu-Wei Fu, Ting-yao Hu, Yu Tsao +1 more

stat.ML · · cited 196× · arXiv ↗ · more like this ·

This paper aims to address two issues existing in the current speech enhancement methods: 1) the difficulty of phase estimations; 2) a single objective function cannot consider multiple metrics simultaneously. To solve the first problem, we propose a novel convolutional neural network (CNN) model for complex spectrogram enhancement, namely estimating clean real and imaginary (RI) spectrograms from noisy ones. The reconstructed RI spectrograms are directly used to synthesize enhanced speech waveforms. In addition, since log-power spectrogram (LPS) can be represented as a function of RI spectrograms, its reconstruction is also considered as another target. Thus a unified objective function, which combines these two targets (reconstruction of RI spectrograms and LPS), is equivalent to simultaneously optimizing two commonly used objective metrics: segmental signal-to-noise ratio (SSNR) and…

Evaluation of Feature Extraction and Recognition for Activity Monitoring and Fall Detection Based on Wearable sEMG Sensors

Xugang Xi, Minyan Tang, Seyed M. Miran +1 more

Computer Vision and Pattern Recognition · Sensors · · cited 142× · DOI ↗ · more like this ·

As an essential subfield of context awareness, activity awareness, especially daily activity monitoring and fall detection, plays a significant role for elderly or frail people who need assistance in their daily activities. This study investigates the feature extraction and pattern recognition of surface electromyography (sEMG), with the purpose of determining the best features and classifiers of sEMG for daily living activities monitoring and fall detection. This is done by a serial of experiments. In the experiments, four channels of sEMG signal from wireless, wearable sensors located on lower limbs are recorded from three subjects while they perform seven activities of daily living (ADL). A simulated trip fall scenario is also considered with a custom-made device attached to the ankle. With this experimental setting, 15 feature extraction methods of sEMG, including time, frequency…

WaveCRN: An Efficient Convolutional Recurrent Neural Network for End-to-end Speech Enhancement

Tsun-An Hsieh, Hsin-Min Wang, Xugang Lu +1 more

eess.AS · · cited 81× · arXiv ↗ · more like this ·

Due to the simple design pipeline, end-to-end (E2E) neural models for speech enhancement (SE) have attracted great interest. In order to improve the performance of the E2E model, the locality and temporal sequential properties of speech should be efficiently taken into account when modelling. However, in most current E2E models for SE, these properties are either not fully considered or are too complex to be realized. In this paper, we propose an efficient E2E SE model, termed WaveCRN. In WaveCRN, the speech locality feature is captured by a convolutional neural network (CNN), while the temporal sequential property of the locality feature is modeled by stacked simple recurrent units (SRU). Unlike a conventional temporal sequential model that uses a long short-term memory (LSTM) network, which is difficult to parallelize, SRU can be efficiently parallelized in calculation with even fewer…

PTfuzz: Guided Fuzzing With Processor Trace Feedback

Gen Zhang, Xu Zhou, Yingqi Luo +2 more

Software · IEEE Access · · cited 70× · DOI ↗ · more like this ·

Greybox fuzzing, such as american fuzzy lop (AFL), is very efficient in finding software vulnerability, which makes it the state-of-the-art fuzzing technology. Greybox fuzzing leverages the branch information collected during program running as feedback to guide choosing seeds. Current greybox fuzzing generally uses two kinds of methods to collect branch information: compile-time instrumentation (AFL) and emulation [AFL extended with QEMU emulation (QAFL)]. Compile-time instrumentation is efficient, but it does not support binary programs. Meanwhile, emulation supports binary programs, but its efficiency is very low. In this paper, we propose a greybox fuzzing approach named PTfuzz, which leverages hardware mechanism (Intel Processor Trace) to collect branch information. Our approach supports binary programs, just like the emulation method, while it gains a comparable performance with…

Speaker Adaptive Training using Deep Neural Networks

Tsubasa Ochiai, Shigeki Matsuda, Xugang Lu +2 more

Artificial Intelligence · · cited 57× · DOI ↗ · more like this ·

Among many speaker adaptation embodiments, Speaker Adaptive Training (SAT) has been successfully applied to a standard Hidden-Markov-Model (HMM) speech recognizer, whose state is associated with Gaussian Mixture Models (GMMs). On the other hand, recent studies on Speaker-Independent (SI) recognizer development have reported that a new type of HMM speech recognizer, which replaces GMMs with Deep Neural Networks (DNNs), outperforms GMM-HMM recognizers. Along these two lines, it is natural to conceive of further improvement to a preset DNN-HMM recognizer by employing SAT. In this paper, we propose a novel training scheme that applies SAT to a SI DNN-HMM recognizer. We then implement the SAT scheme by allocating a Speaker-Dependent (SD) module to one of the intermediate layers of a seven-layer DNN, and elaborate its utility over TED Talks corpus data. Experiment results show that our…

Speech enhancement using segmental nonnegative matrix factorization

Hao-teng Fan, Jeih-weih Hung, Xugang Lu +2 more

Signal Processing · · cited 49× · DOI ↗ · more like this ·

The conventional NMF-based speech enhancement algorithm analyzes the magnitude spectrograms of both clean speech and noise in the training data via NMF and estimates a set of spectral basis vectors. These basis vectors are used to span a space to approximate the magnitude spectrogram of the noise-corrupted testing utterances. Finally, the components associated with the clean-speech spectral basis vectors are used to construct the updated magnitude spectrogram, producing an enhanced speech utterance. Considering that the rich spectral-temporal structure may be explored in local frequency and time-varying spectral patches, this study proposes a segmental NMF (SNMF) speech enhancement scheme to improve the conventional frame-wise NMF-based method. Two algorithms are derived to decompose the original nonnegative matrix associated with the magnitude spectrogram; the first algorithm is used…

Speech Enhancement based on Denoising Autoencoder with Multi-branched Encoders

Cheng Yu, Ryandhimas E. Zezario, Syu-Siang Wang +5 more

eess.AS · · cited 47× · arXiv ↗ · more like this ·

Deep learning-based models have greatly advanced the performance of speech enhancement (SE) systems. However, two problems remain unsolved, which are closely related to model generalizability to noisy conditions: (1) mismatched noisy condition during testing, i.e., the performance is generally sub-optimal when models are tested with unseen noise types that are not involved in the training data; (2) local focus on specific noisy conditions, i.e., models trained using multiple types of noises cannot optimally remove a specific noise type even though the noise type has been involved in the training data. These problems are common in real applications. In this paper, we propose a novel denoising autoencoder with a multi-branched encoder (termed DAEME) model to deal with these two problems. In the DAEME model, two stages are involved: training and testing. In the training stage, we build…

Sparse representation based on a bag of spectral exemplars for acoustic event detection

Xugang Lu, Yu Tsao, Shigeki Matsuda +1 more

Signal Processing · · cited 33× · DOI ↗ · more like this ·

Acoustic event detection is an important step for audio content analysis and retrieval. Traditional detection techniques model the acoustic events on frame-based spectral features. Considering the temporal-frequency structures of acoustic events may be distributed in time-scales beyond frames, we propose to represent those structures as a bag of spectral patch exemplars. In order to learn the representative exemplars, k-means clustering based vector quantization (VQ) was applied on the whitened spectral patches which makes the learned exemplars focus on high-order statistical structure. With the learned spectral exemplars, a sparse feature representation is extracted based on the similarity measurement to the learned exemplars. A support vector machine (SVM) classifier was built on the sparse representation for acoustic event detection. Our experimental results showed that the sparse…

A 6-DOF robot-time optimal trajectory planning based on an improved genetic algorithm

Jiayan Zhang, Qingxi Meng, Xugang Feng +1 more

Computer Vision and Pattern Recognition · Robotics and Biomimetics · · cited 32× · DOI ↗ · more like this ·

By using quintic polynomial function to interpolate several given points of each joint of the robot, the mathematical expressions of each joint variable of the robot with time are established. In addition, to improve the search algorithm performance crossover operator and mutation operator of the genetic algorithm are improved in cosine form. Furthermore, the improved adaptive genetic algorithm is applied to optimize the time interval of interpolation points of each joint, so as to realize time optimal trajectory planning. Moreover, MATLAB simulation is carried out, and the results show that the method proposed in this paper reduces the running time of the robot tasks. Meanwhile, the curves of angle position, velocity and acceleration of each joint are smooth enough, which ensure accomplish its tasks in a stable and efficient way.

Factored Language Model based on Recurrent Neural Network

Youzheng Wu, Xugang Lu, H. Yamamoto +3 more

Artificial Intelligence · International Conference on Computational Linguistics · · cited 31× · OpenAlex ↗ · more like this ·

Among various neural network language models (NNLMs), recurrent neural network-based language models (RNNLMs) are very competitive in many cases. Most current RNNLMs only use one single feature stream, i.e., surface words. However, previous studies proved that language models with additional linguistic information achieve better performance. In this study, we extend RNNLM by explicitly integrating additional linguistic information, including morphological, syntactic, or semantic factors. Our proposed RNNLM is called a factored RNNLM that is expected to enhance RNNLMs. A number of experiments are carried out that show the factored RNNLM improves the performance for all considered tasks: consistent perplexity and word error rate (WER) reductions. In the Penn Treebank corpus, the relative improvements over n-gram LM and RNNLM are 29.0 % and 13.0%, respectively. In the IWSLT-2011 TED ASR…

Physiological Feature Extraction for Text Independent Speaker Identification using Non-Uniform Subband Processing

Xugang Lu, Jianwu Dang

Artificial Intelligence · · cited 27× · DOI ↗ · more like this ·

The features used for speech recognition should emphasize linguistic information while suppressing speaker differences. For speaker recognition, features should have more speaker individual information while attenuating the linguistic information. In most studies, however, the identical acoustic features are used for the different missions of speaker and speech recognitions. In this paper, we propose a new physiological feature extraction method which emphasizes individual information for speaker identification. For the purpose, physiological features of speakers were analyzed from the point of view of speech production. It is found that the speaker individual information is encoded in different frequency regions of speech sound. The speaker discriminative information was quantified using Fisher's F-ratio in each frequency region. Based on the F-ratio, we proposed a non-uniform sub-band…

Knowledge Distillation-Based Representation Learning for Short-Utterance Spoken Language Identification

Peng Shen, Xugang Lu, Sheng Li +1 more

Artificial Intelligence · IEEE/ACM Transactions on Audio Speech and Language Processing · · cited 26× · DOI ↗ · more like this ·

With successful applications of deep feature learning algorithms, spoken language identification (LID) on long utterances obtains satisfactory performance. However, the performance on short utterances is drastically degraded even when the LID system is trained using short utterances. The main reason is due to the large variation of the representation on short utterances which results in high model confusion. To narrow the performance gap between long, and short utterances, we proposed a teacher-student representation learning framework based on a knowledge distillation method to improve LID performance on short utterances. In the proposed framework, in addition to training the student model on short utterances with their true labels, the internal representation from the output of a hidden layer of the student model is supervised with the representation corresponding to their longer…

Self-Supervised Denoising Autoencoder with Linear Regression Decoder for Speech Enhancement

Ryandhimas E. Zezario, Tassadaq Hussain, Xugang Lu +2 more

Signal Processing · · cited 25× · DOI ↗ · more like this ·

Nonlinear spectral mapping-based models based on supervised learning have successfully applied for speech enhancement. However, as supervised learning approaches, a large amount of labelled data (noisy-clean speech pairs) should be provided to train those models. In addition, their performances for unseen noisy conditions are not guaranteed, which is a common weak point of supervised learning approaches. In this study, we proposed an unsupervised learning approach for speech enhancement, i.e., denoising autoencoder with linear regression decoder (DAELD) model for speech enhancement. The DAELD is trained with noisy speech as both input and target output in a self-supervised learning manner. In addition, with properly setting a shrinkage threshold for internal hidden representations, noise could be removed during the reconstruction from the hidden representations via the linear regression…

Unsupervised neural adaptation model based on optimal transport for spoken language identification

Xugang Lu, Peng Shen, Yu Tsao +1 more

cs.LG · · cited 24× · arXiv ↗ · more like this ·

Due to the mismatch of statistical distributions of acoustic speech between training and testing sets, the performance of spoken language identification (SLID) could be drastically degraded. In this paper, we propose an unsupervised neural adaptation model to deal with the distribution mismatch problem for SLID. In our model, we explicitly formulate the adaptation as to reduce the distribution discrepancy on both feature and classifier for training and testing data sets. Moreover, inspired by the strong power of the optimal transport (OT) to measure distribution discrepancy, a Wasserstein distance metric is designed in the adaptation loss. By minimizing the classification loss on the training data set with the adaptation loss on both training and testing data sets, the statistical distribution difference between training and testing domains is reduced. We carried out SLID experiments on…

Perceptual Contrast Stretching on Target Feature for Speech Enhancement

Rong Chao, Cheng Yu, Szu-Wei Fu +2 more

cs.SD · · cited 22× · arXiv ↗ · more like this ·

Speech enhancement (SE) performance has improved considerably owing to the use of deep learning models as a base function. Herein, we propose a perceptual contrast stretching (PCS) approach to further improve SE performance. The PCS is derived based on the critical band importance function and is applied to modify the targets of the SE model. Specifically, the contrast of target features is stretched based on perceptual importance, thereby improving the overall SE performance. Compared with post-processing-based implementations, incorporating PCS into the training phase preserves performance and reduces online computation. Notably, PCS can be combined with different SE model architectures and training criteria. Furthermore, PCS does not affect the causality or convergence of SE model training. Experimental results on the VoiceBank-DEMAND dataset show that the proposed method can achieve…

Contributions of Jitter and Shimmer in the Voice for Fake Audio Detection

Li Kai, Xugang Lu, Masato Akagi +1 more

Signal Processing · IEEE Access · · cited 20× · DOI ↗ · more like this ·

Fake audio detection (FAD) aims to identify fraudulent speech generated through advanced speech-synthesis techniques. Most current FAD methods rely solely on a deep neural network (DNN) framework with either speech waveforms or commonly used acoustic features to extract high-level representations, overlooking the analysis of prosody differences between genuine and fake speech. Prosody carries important cues about the naturalness of speech and emotional content, which can be leveraged in the detection of fake audio. This paper explicitly investigates the differences in prosody information between genuine and fake speech represented by the jitter and shimmer features. On the basis of our investigation, we found strong evidence that obvious differences exist in the level of jitter and shimmer between fake and real speech, particularly on the shimmer feature that has a large dynamic…

Wavelet speech enhancement based on nonnegative matrix factorization

Syu-Siang Wang, Alan Chern, Yu Tsao +4 more

cs.SD · · cited 19× · arXiv ↗ · more like this ·

For most of the state-of-the-art speech enhancement techniques, a spectrogram is usually preferred than the respective time-domain raw data since it reveals more compact presentation together with conspicuous temporal information over a long time span. However, the short-time Fourier transform (STFT) that creates the spectrogram in general distorts the original signal and thereby limits the capability of the associated speech enhancement techniques. In this study, we propose a novel speech enhancement method that adopts the algorithms of discrete wavelet packet transform (DWPT) and nonnegative matrix factorization (NMF) in order to conquer the aforementioned limitation. In brief, the DWPT is first applied to split a time-domain speech signal into a series of subband signals without introducing any distortion. Then we exploit NMF to highlight the speech component for each subband.…

Incorporating Symbolic Sequential Modeling for Speech Enhancement

Chien-Feng Liao, Yu Tsao, Xugang Lu +1 more

cs.LG · · cited 18× · arXiv ↗ · more like this ·

In a noisy environment, a lossy speech signal can be automatically restored by a listener if he/she knows the language well. That is, with the built-in knowledge of a "language model", a listener may effectively suppress noise interference and retrieve the target speech signals. Accordingly, we argue that familiarity with the underlying linguistic content of spoken utterances benefits speech enhancement (SE) in noisy environments. In this study, in addition to the conventional modeling for learning the acoustic noisy-clean speech mapping, an abstract symbolic sequential modeling is incorporated into the SE framework. This symbolic sequential modeling can be regarded as a "linguistic constraint" in learning the acoustic noisy-clean speech mapping function. In this study, the symbolic sequences for acoustic signals are obtained as discrete representations with a Vector Quantized…

Research on Identification and Location of Mining Landslide in Mining Area Based on Improved YOLO Algorithm

Xugang Lian, Li Yu, Xiaobing Wang +2 more

Computer Vision and Pattern Recognition · Drones · · cited 17× · DOI ↗ · more like this ·

The wide range and high intensity of landslides in the mining area pose a great threat to the safety of human life and property. It is particularly important to identify and monitor them. However, due to the serious surface damage, small landslide scale, complex background and other factors in the mining area, it is impossible to accurately identify and detect the landslide in the mining area. It is necessary to select an efficient detection model to detect it. In this paper, aiming at the problem of landslide identification in mining area, the remote sensing image of mining area is obtained by unmanned aerial vehicle (UAV), and the landslide data set of mining area is constructed by data enhancement method. An improved YOLOv8 algorithm is proposed. By adding a mixed attention mechanism in the channel and spatial dimensions, the detection accuracy of the model for mining landslide is…

Semi-supervised ensemble DNN acoustic model training

Sheng Li, Xugang Lu, Shinsuke Sakai +2 more

Artificial Intelligence · Tokyo Tech Research Repository (Tokyo Institute of Technology) · · cited 17× · DOI ↗ · more like this ·

It is very important to exploit abundant unlabeled speech for improving the acoustic model training in automatic speech recognition (ASR). Semi-supervised training methods incorporate unlabeled data in addition to labeled data to enhance the model training, but it encounters the error-prone label problem. The ensemble training scheme trains a set of models and combines them to make the model more general and robust, but it has not been applied to the unlabeled data. In this work, we propose an effective semi-supervised training of deep neural network (DNN) acoustic models by incorporating the diversity among the ensemble of models. The resultant model improved the performance in the lecture transcription task. Moreover, the proposed method has also shown a potential for DNN adaptation.

Improving Perceptual Quality by Phone-Fortified Perceptual Loss using Wasserstein Distance for Speech Enhancement

Tsun-An Hsieh, Cheng Yu, Szu-Wei Fu +2 more

cs.SD · · cited 16× · arXiv ↗ · more like this ·

Speech enhancement (SE) aims to improve speech quality and intelligibility, which are both related to a smooth transition in speech segments that may carry linguistic information, e.g. phones and syllables. In this study, we propose a novel phone-fortified perceptual loss (PFPL) that takes phonetic information into account for training SE models. To effectively incorporate the phonetic information, the PFPL is computed based on latent representations of the wav2vec model, a powerful self-supervised encoder that renders rich phonetic information. To more accurately measure the distribution distances of the latent representations, the PFPL adopts the Wasserstein distance as the distance measure. Our experimental results first reveal that the PFPL is more correlated with the perceptual evaluation metrics, as compared to signal-level losses. Moreover, the results showed that the PFPL can…