arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Tian Tan

264 papers · 1998–2026

1998: 11999: 12000: 22001: 52002: 72003: 122004: 52005: 62006: 92007: 52008: 32009: 52010: 52011: 42012: 72013: 162014: 52015: 82016: 142017: 102018: 212019: 102020: 82021: 182022: 122023: 162024: 142025: 102026: 24
1998peak 24 in 20262026

Frequent co-authors

Papers

showing 1–25 of 264 RSS

SALMONN: Towards Generic Hearing Abilities for Large Language Models

Changli Tang, Wenyi Yu, Guangzhi Sun +6 more

cs.SD · · cited 719× · arXiv ↗ · more like this ·

Hearing is arguably an essential ability of artificial intelligence (AI) agents in the physical world, which refers to the perception and understanding of general auditory information consisting of at least three types of sounds: speech, audio events, and music. In this paper, we propose SALMONN, a speech audio language music open neural network, built by integrating a pre-trained text-based large language model (LLM) with speech and audio encoders into a single multimodal model. SALMONN enables the LLM to directly process and understand general audio inputs and achieve competitive performances on a number of speech and audio tasks used in training, such as automatic speech recognition and translation, auditory-information-based question answering, emotion recognition, speaker verification, and music and audio captioning etc. SALMONN also has a diverse set of emergent abilities unseen…

Very Deep Convolutional Neural Networks for Noise Robust Speech Recognition

Yanmin Qian, Mengxiao Bi, Tian Tan +1 more

Artificial Intelligence · IEEE/ACM Transactions on Audio Speech and Language Processing · · cited 347× · DOI ↗ · more like this ·

Although great progress has been made in automatic speech recognition, significant performance degradation still exists in noisy environments. Recently, very deep convolutional neural networks (CNNs) have been successfully applied to computer vision and speech recognition tasks. Based on our previous work on very deep CNNs, in this paper this architecture is further developed to improve recognition accuracy for noise robust speech recognition. In the proposed very deep CNN architecture, we study the best configuration for the sizes of filters, pooling, and input feature maps: the sizes of filters and poolings are reduced and dimensions of input features are extended to allow for adding more convolutional layers. Then the appropriate pooling, padding, and input feature map selection strategies are investigated and applied to the very deep CNN to make it more robust for speech…

On multiple description streaming with content delivery networks

J Apostolopoulos, Tina Wong, Wai-Tian Tan +1 more

Computer Networks and Communications · · cited 327× · DOI ↗ · more like this ·

We propose a system that improves the performance of streaming media CDN by exploiting the path diversity provided by existing CDN infrastructure. Path diversity is provided by the different network paths that exist between a client and its nearby edge servers; and multiple description (MD) coding is coupled with this path diversity to provide resilience to losses. In our system, MD coding is used to code a media stream into multiple complementary descriptions, which are distributed across the edge servers in the CDN. When a client requests a media stream, it is directed to multiple nearby servers which host complementary descriptions. These servers simultaneously stream these complementary descriptions to the client over different network paths. This paper provides distortion models for MDC video and conventional video. We use these models to select the optimal pair of servers with…

Real-time Internet video using error resilient scalable compression and TCP-friendly transport protocol

Wai-Tian Tan, Avideh Zakhor

Signal Processing · IEEE Transactions on Multimedia · · cited 281× · DOI ↗ · more like this ·

We introduce a point to point real-time video transmission scheme over the Internet combining a low-delay TCP-friendly transport protocol in conjunction with a novel compression method that is error resilient and bandwidth-scalable. Compressed video is packetized into individually decodable packets of equal expected visual importance. Consequently, relatively constant video quality can be achieved at the receiver under lossy conditions. Furthermore, the packets can be truncated to instantaneously meet the time varying bandwidth imposed by a TCP-friendly transport protocol. As a result, adaptive flows that are friendly to other Internet traffic are produced. Actual Internet experiments together with simulations are used to evaluate the performance of the compression, transport, and the combined schemes.

Video multicast using layered FEC and scalable compression

Wai-Tian Tan, Avideh Zakhor

Computer Networks and Communications · IEEE Transactions on Circuits and Systems for Video Technology · · cited 215× · DOI ↗ · more like this ·

The use of scalable video with layered multicast has been shown to be an effective method to achieve rate control in heterogeneous networks. We propose the use of layered forward error correction (FEC) as an error-control mechanism in a layered multicast framework. By organizing FEC into multiple layers, receivers can obtain different levels of protection commensurate with their respective channel conditions. Efficient network utilization is achieved as FEC streams are multicast, and only to receivers that need them. Furthermore, FEC is used without overall rate expansion by selectively dropping data layers to make room for FEC layers. Effects of bursty losses are amortized by staggering the FEC streams in time, giving rise to a tradeoff between delay and quality. For rate control at the receivers, we propose an equation-based approach that computes network usage as a function of…

Connecting Speech Encoder and Large Language Model for ASR

Wenyi Yu, Changli Tang, Guangzhi Sun +6 more

eess.AS · · cited 139× · arXiv ↗ · more like this ·

The impressive capability and versatility of large language models (LLMs) have aroused increasing attention in automatic speech recognition (ASR), with several pioneering studies attempting to build integrated ASR models by connecting a speech encoder with an LLM. This paper presents a comparative study of three commonly used structures as connectors, including fully connected layers, multi-head cross-attention, and Q-Former. Speech encoders from the Whisper model series as well as LLMs from the Vicuna model series with different model sizes were studied. Experiments were performed on the commonly used LibriSpeech, Common Voice, and GigaSpeech datasets, where the LLMs with Q-Formers demonstrated consistent and considerable word error rate (WER) reductions over LLMs with other connector structures. Q-Former-based LLMs can generalise well to out-of-domain datasets, where 12% relative WER…

Reshaping deep neural network for fast decoding by node-pruning

Tianxing He, Yuchen Fan, Yanmin Qian +2 more

Artificial Intelligence · · cited 117× · DOI ↗ · more like this ·

Although deep neural networks (DNN) has achieved significant accuracy improvements in speech recognition, it is computationally expensive to deploy large-scale DNN in decoding due to huge number of parameters. Weights truncation and decomposition methods have been proposed to speed up decoding by exploiting the sparseness of DNN. This paper summarizes different approaches of restructuring DNN and proposes a new node pruning approach to reshape DNN for fast decoding. In this approach, hidden nodes of a fully trained DNN are pruned with certain importance function and the reshaped DNN is retuned using back-propagation. The approach requires no modification on code and can directly save computational costs during decoding. Furthermore, it is complementary to weight decomposition methods. Experiments on a switchboard task shows that, by using the proposed node-pruning approach, DNN…

Layered Constructions for Low-Delay Streaming Codes

Ahmed Badr, Pratik Patil, Ashish Khisti +2 more

cs.IT · · cited 117× · arXiv ↗ · more like this ·

We propose a new class of error correction codes for low-delay streaming communication. We consider an online setup where a source packet arrives at the encoder every $M$ channel uses, and needs to be decoded with a maximum delay of $T$ packets. We consider a sliding-window erasure channel --- $\cC(N,B,W)$ --- which introduces either up to $N$ erasures in arbitrary positions, or $B$ erasures in a single burst, in any window of length $W$. When $M=1$, the case where source-arrival and channel-transmission rates are equal, we propose a class of codes --- MiDAS codes --- that achieve a near optimal rate. Our construction is based on a {\em layered} approach. We first construct an optimal code for the $\cC(N=1,B,W)$ channel, and then concatenate an additional layer of parity-check symbols to deal with $N>1$. When $M > 1$, the case where source-arrival and channel-transmission rates are…

Generating Adjacency-Constrained Subgoals in Hierarchical Reinforcement Learning

Tianren Zhang, Shangqi Guo, Tian Tan +2 more

cs.LG · · cited 110× · arXiv ↗ · more like this ·

Goal-conditioned hierarchical reinforcement learning (HRL) is a promising approach for scaling up reinforcement learning (RL) techniques. However, it often suffers from training inefficiency as the action space of the high-level, i.e., the goal space, is often large. Searching in a large goal space poses difficulties for both high-level subgoal generation and low-level policy learning. In this paper, we show that this problem can be effectively alleviated by restricting the high-level action space from the whole goal space to a $k$-step adjacent region of the current state using an adjacency constraint. We theoretically prove that the proposed adjacency constraint preserves the optimal hierarchical policy in deterministic MDPs, and show that this constraint can be practically implemented by training an adjacency network that can discriminate between adjacent and non-adjacent subgoals.…

Postprocessing method for suppressing musical noise generated by spectral subtraction

Zenton Goh, Kah-Chye Tan, Tian Tan

Signal Processing · IEEE Transactions on Speech and Audio Processing · · cited 96× · DOI ↗ · more like this ·

We investigate whether musical noise, which often exists in speech enhanced using spectral subtraction, can be suppressed. Via exploiting some specific characteristics of human speech, we propose a method that can effectively suppress musical noise without a noticeable effect on speech intelligibility. Performance assessments confirm that our method is effective.

Auditing Anti-Malware Tools by Evolving Android Malware and Dynamic Loading Technique

Yinxing Xue, Guozhu Meng, Yang Liu +4 more

Signal Processing · IEEE Transactions on Information Forensics and Security · · cited 92× · DOI ↗ · more like this ·

Although a previous paper shows that existing anti-malware tools (AMTs) may have high detection rate, the report is based on existing malware and thus it does not imply that AMTs can effectively deal with future malware. It is desirable to have an alternative way of auditing AMTs. In our previous paper, we use malware samples from android malware collection Genome to summarize a malware meta-model for modularizing the common attack behaviors and evasion techniques in reusable features. We then combine different features with an evolutionary algorithm, in which way we evolve malware for variants. Previous results have shown that the existing AMTs only exhibit detection rate of 20%-30% for 10 000 evolved malware variants. In this paper, based on the modularized attack features, we apply the dynamic code generation and loading techniques to produce malware, so that we can audit the AMTs at…

Recurrent neural network language model adaptation for multi-genre broadcast speech recognition

X. Chen, Tian Tan, Xunying Liu +4 more

Artificial Intelligence · Cambridge University Engineering Department Publications Database · · cited 91× · DOI ↗ · more like this ·

Recurrent neural network language models (RNNLMs) have recently become increasingly popular for many applications including speech recognition. In previous research RNNLMs have normally been trained on well-matched in-domain data. The adaptation of RNNLMs remains an open research area to be explored. In this paper, genre and topic based RNNLMadaptation techniques are investigated for a multi-genre broadcast transcription task. A number of techniques including Probabilistic Latent Semantic Analysis, Latent Dirichlet Allocation and Hierarchical Dirichlet Processes are used to extract show level topic information. These were then used as additional input to the RNNLM during training, which can facilitate unsupervised test time adaptation. Experiments using a state-of-theart LVCSR system trained on 1000 hours of speech and more than 1 billion words of text showed adaptation could yield…

Modeling path diversity for multiple description video communication

John Apostolopoulos, Wai-Tian Tan, Susie Wee +1 more

Computer Networks and Communications · IEEE International Conference on Acoustics Speech and Signal Processing · · cited 84× · DOI ↗ · more like this ·

The use of multiple description (MD) video coding and path diversity has been proposed to provide improved performance over lossy packet networks [1]. The goal of this work was to develop models to accurately and quickly predict and compare the distortion of MD video coding and path diversity against conventional single description (SD) video delivered over a single path. In the process, we developed (1) a model for the loss process of a two-path path diversity system, and (2) a distortion model that maps the loss model to MD distortion values. Given these models we present a number of comparisons between MD video coding and path diversity and conventional SD video over a single path. The proposed model for path diversity may also be useful in other applications not related to MD coding. Furthermore, other forms of MD coding may be analyzed using similar models for MD distortion.

Efficient and precise points-to analysis: modeling the heap by merging equivalent automata

Tian Tan, Yue Li, Jingling Xue

Signal Processing · · cited 80× · DOI ↗ · more like this ·

Mainstream points-to analysis techniques for object-oriented languages rely predominantly on the allocation-site abstraction to model heap objects. We present MAHJONG, a novel heap abstraction that is specifically developed to address the needs of an important class of type-dependent clients, such as call graph construction, devirtualization and may-fail casting. By merging equivalent automata representing type-consistent objects that are created by the allocation-site abstraction, MAHJONG enables an allocation-site-based points-to analysis to run significantly faster while achieving nearly the same precision for type-dependent clients.

Precision-guided context sensitivity for pointer analysis

Yue Li, Tian Tan, Anders Møller +1 more

Signal Processing · Proceedings of the ACM on Programming Languages · · cited 79× · DOI ↗ · more like this ·

Context sensitivity is an essential technique for ensuring high precision in Java pointer analyses. It has been observed that applying context sensitivity partially, only on a select subset of the methods, can improve the balance between analysis precision and speed. However, existing techniques are based on heuristics that do not provide much insight into what characterizes this method subset. In this work, we present a more principled approach for identifying precision-critical methods, based on general patterns of value flows that explain where most of the imprecision arises in context-insensitive pointer analysis. Accordingly, we provide an efficient algorithm to recognize these flow patterns in a given program and exploit them to yield good tradeoffs between analysis precision and speed. Our experimental results on standard benchmark and real-world programs show that a pointer…

Cluster adaptive training for deep neural network

Tian Tan, Yanmin Qian, Maofan Yin +2 more

Artificial Intelligence · · cited 71× · DOI ↗ · more like this ·

Although context-dependent DNN-HMM systems have achieved significant improvements over GMM-HMM systems, there still exists big performance degradation if the acoustic condition of the test data mismatches that of the training data. Hence, adaptation and adaptive training of DNN are of great research interest. Previous works mainly focus on adapting the parameters of a single DNN by regularized or selective fine-tuning, applying linear transforms to feature or hidden-layer output, or introducing vector representation of non-speech variability into the input. These methods all require relatively large number of parameters to be estimated during adaptation. In contrast, this paper employs the cluster adaptive training (CAT) framework for DNN adaptation. Here, multiple DNNs are constructed to form the bases of a canonical parametric space. During adaptation, an interpolation vector…

Understanding and Analyzing Java Reflection

Yue Li, Tian Tan, Jingling Xue

cs.PL · · cited 69× · arXiv ↗ · more like this ·

Java reflection has been increasingly used in a wide range of software. It allows a software system to inspect and/or modify the behaviour of its classes, interfaces, methods and fields at runtime, enabling the software to adapt to dynamically changing runtime environments. However, this dynamic language feature imposes significant challenges to static analysis, because the behaviour of reflection-rich software is logically complex and statically hard to predict, especially when manipulated frequently by statically unknown string values. As a result, existing static analysis tools either ignore reflection or handle it partially, resulting in missed, important behaviours, i.e., unsound results. Therefore, improving or even achieving soundness in (static) reflection analysis -- an analysis that infers statically the behaviour of reflective code -- will provide significant benefits to many…

Adaptive Very Deep Convolutional Residual Network for Noise Robust Speech Recognition

Tian Tan, Yanmin Qian, Hu Hu +3 more

Artificial Intelligence · IEEE/ACM Transactions on Audio Speech and Language Processing · · cited 65× · DOI ↗ · more like this ·

Although great progress has been made in automatic speech recognition, significant performance degradation still exists in noisy environments. Our previous work has demonstrated the superior noise robustness of very deep convolutional neural networks (VDCNN). Based on our work on VDCNNs, this paper proposes a more advanced model referred to as the very deep convolutional residual network (VDCRN). This new model incorporates batch normalization and residual learning, showing more robustness than previous VDCNNs.Then, to alleviate the mismatch between the training and testing conditions, model adaptation and adaptive training are developed and compared for the new VDCRN. This paper focuses on factor aware training (FAT) and cluster adaptive training (CAT). For FAT, a unified framework is explored. For CAT, two schemes are first explored to construct the bases in the canonical model…

Scalability-first pointer analysis with self-tuning context-sensitivity

Yue Li, Tian Tan, Anders Møller +1 more

Signal Processing · · cited 64× · DOI ↗ · more like this ·

Context-sensitivity is important in pointer analysis to ensure high precision, but existing techniques suffer from unpredictable scalability. Many variants of context-sensitivity exist, and it is difficult to choose one that leads to reasonable analysis time and obtains high precision, without running the analysis multiple times.

Streaming Codes for Channels with Burst and Isolated Erasures

Ahmed Badr, Ashish Khisti, Wai-Tian Tan +1 more

cs.IT · · cited 63× · arXiv ↗ · more like this ·

We study low-delay error correction codes for streaming recovery over a class of packet-erasure channels that introduce both burst-erasures and isolated erasures. We propose a simple, yet effective class of codes whose parameters can be tuned to obtain a tradeoff between the capability to correct burst and isolated erasures. Our construction generalizes previously proposed low-delay codes which are effective only against burst erasures. We establish an information theoretic upper bound on the capability of any code to simultaneously correct burst and isolated erasures and show that our proposed constructions meet the upper bound in some special cases. We discuss the operational significance of column-distance and column-span metrics and establish that the rate 1/2 codes discovered by Martinian and Sundberg [IT Trans.\, 2004] through a computer search indeed attain the optimal…

A Principled Approach to Selective Context Sensitivity for Pointer Analysis

Yue Li, Tian Tan, Anders Møller +1 more

Information Systems · ACM Transactions on Programming Languages and Systems · · cited 60× · DOI ↗ · more like this ·

Context sensitivity is an essential technique for ensuring high precision in static analyses. It has been observed that applying context sensitivity partially, only on a select subset of the methods, can improve the balance between analysis precision and speed. However, existing techniques are based on heuristics that do not provide much insight into what characterizes this method subset. In this work, we present a more principled approach for identifying precision-critical methods, based on general patterns of value flows that explain where most of the imprecision arises in context-insensitive pointer analysis. Using this theoretical foundation, we present an efficient algorithm, Z IPPER , to recognize these flow patterns in a given program and employ context sensitivity accordingly. We also present a variant, Z IPPER e , that additionally takes into account which methods are…

Generative Adversarial Networks Based Data Augmentation for Noise Robust Speech Recognition

Hu Hu, Tian Tan, Yanmin Qian

Signal Processing · · cited 56× · DOI ↗ · more like this ·

Data augmentation is an effective method to increase the size of training data and reduce the mismatch between training and testing for noise robust speech recognition. Different from the traditional approaches by directly adding noise to the original waveform, in this work we utilize generative adversarial networks (GAN) for data generation to improve speech recognition under noise conditions. With this method, the generated speech samples are based on spectrum feature level and produced frame by frame without dependence among them, and the augmented data has no true labels. Then to effectively use these untranscribed augmented data, an unsupervised learning framework is designed for acoustic modeling. The proposed GAN-based data augmentation approach is evaluated on Aurora4. The experimental results show that a relative ~ 7.0% WER reduction can be obtained by the proposed approach…

Perfecting Protection for Interactive Multimedia: A survey of forward error correction for low-delay interactive applications

Ahmed Badr, Ashish Khisti, Wai-Tian Tan +1 more

Computer Networks and Communications · IEEE Signal Processing Magazine · · cited 52× · DOI ↗ · more like this ·

Many current and emerging applications require low-latency communication, including interactive voice and video communication, multiplayer gaming, multiperson augmented/virtual reality, and various Internet of Things (IoT) applications. Forward error correction (FEC) codes for low-delay interactive applications have several characteristics that distinguish them from traditional FEC. The encoding and decoding operations must process a stream of data packets in a sequential fashion. Strict latency constraints limit the use of long block lengths, interleaving, or large buffers. Furthermore, these codes must achieve fast recovery from burst losses and yet be robust to other types of loss patterns.

Cluster Adaptive Training for Deep Neural Network Based Acoustic Model

Tian Tan, Yanmin Qian, Kai Yu

Artificial Intelligence · IEEE/ACM Transactions on Audio Speech and Language Processing · · cited 52× · DOI ↗ · more like this ·

Although context-dependent DNN-HMM systems have achieved significant improvements over GMM-HMM systems, significant performance degradation has been observed if the acoustic condition of the test data mismatches that of the training data. Hence, adaptation and adaptive training of DNN are of great research interest. Previous DNN adaptation works mainly focus on adapting parameters of a single DNN by applying linear transformations to feature or hidden-layer output; introducing vector representation of non-speech variability into the input. In these methods, large number of parameters are required to be estimated during adaptation. In this paper, the cluster adaptive training (CAT) framework is employed for DNN adaptive training. Here, multiple weight matrices are constructed to form the basis of a canonical parametric space. During adaptation, for a new acoustic condition, an…