arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Jun Lin

716 papers · 1996–2026

1996: 41997: 21998: 21999: 02000: 22001: 42002: 162003: 72004: 82005: 112006: 172007: 202008: 202009: 192010: 232011: 222012: 312013: 362014: 332015: 272016: 262017: 202018: 422019: 472020: 462021: 342022: 282023: 422024: 502025: 472026: 30
1996peak 50 in 20242026

Frequent co-authors

Papers

showing 1–25 of 716 RSS

Intrusion Detection of Imbalanced Network Traffic Based on Machine Learning and Deep Learning

Lan Liu, Pengcheng Wang, Jun Lin +1 more

Computer Networks and Communications · IEEE Access · · cited 257× · DOI ↗ · more like this ·

In imbalanced network traffic, malicious cyber-attacks can often hide in large amounts of normal data. It exhibits a high degree of stealth and obfuscation in cyberspace, making it difficult for Network Intrusion Detection System(NIDS) to ensure the accuracy and timeliness of detection. This paper researches machine learning and deep learning for intrusion detection in imbalanced network traffic. It proposes a novel Difficult Set Sampling Technique(DSSTE) algorithm to tackle the class imbalance problem. First, use the Edited Nearest Neighbor(ENN) algorithm to divide the imbalanced training set into the difficult set and the easy set. Next, use the KMeans algorithm to compress the majority samples in the difficult set to reduce the majority. Zoom in and out the minority samples' continuous attributes in the difficult set synthesize new samples to increase the minority number. Finally…

A High-Speed and Low-Complexity Architecture for Softmax Function in Deep Learning

Meiqi Wang, Siyuan Lu, Danyang Zhu +2 more

Computer Vision and Pattern Recognition · · cited 196× · DOI ↗ · more like this ·

Recently, significant improvement has been achieved for hardware architecture design of deep neural networks (DNNs). However, the hardware implementation of one widely used softmax function in DNNs has not been much investigated, which involves expensive division and exponentiation units. This paper performs an efficient hardware implementation of softmax function. Mathematical transformations and linear fitting are used to simplify this function. Multiple algorithmic strength reduction strategies and fast addition methods are employed to optimize the architecture. By using these techniques, complicated logic units like multipliers are eliminated and the memory consumption is largely reduced while the accuracy loss is negligible. The proposed design is coded using hardware description language (HDL) and synthesized under the TSMC 28-nm CMOS technology. Synthesis results show that the…

Hardware Accelerator for Multi-Head Attention and Position-Wise Feed-Forward in the Transformer

Siyuan Lu, Meiqi Wang, Shuang Liang +2 more

eess.SP · · cited 153× · arXiv ↗ · more like this ·

Designing hardware accelerators for deep neural networks (DNNs) has been much desired. Nonetheless, most of these existing accelerators are built for either convolutional neural networks (CNNs) or recurrent neural networks (RNNs). Recently, the Transformer model is replacing the RNN in the natural language processing (NLP) area. However, because of intensive matrix computations and complicated data flow being involved, the hardware design for the Transformer model has never been reported. In this paper, we propose the first hardware accelerator for two key components, i.e., the multi-head attention (MHA) ResBlock and the position-wise feed-forward network (FFN) ResBlock, which are the two most complex layers in the Transformer. Firstly, an efficient method is introduced to partition the huge matrices in the Transformer, allowing the two ResBlocks to share most of the hardware resources.…

Efficient Hardware Architectures for Deep Convolutional Neural Network

Jichen Wang, Jun Lin, Zhongfeng Wang

Computer Vision and Pattern Recognition · IEEE Transactions on Circuits and Systems I Regular Papers · · cited 144× · DOI ↗ · more like this ·

Convolutional neural network (CNN) is the state-of-the-art deep learning approach employed in various applications. Real-time CNN implementations in resource limited embedded systems are becoming highly desired recently. To ensure the programmable flexibility and shorten the development period, field programmable gate array is appropriate to implement the CNN models. However, the limited bandwidth and on-chip memory storage are the bottlenecks of the CNN acceleration. In this paper, we propose efficient hardware architectures to accelerate deep CNN models. The theoretical derivation of parallel fast finite impulse response algorithm (FFA) is introduced. Based on FFAs, the corresponding fast convolution units (FCUs) are developed for the computation of convolutions in the CNN models. Novel data storage and reuse schemes are proposed, where all intermediate pixels are stored on-chip and…

Efficient Precision-Adjustable Architecture for Softmax Function in Deep Learning

Danyang Zhu, Siyuan Lu, Meiqi Wang +2 more

Computer Vision and Pattern Recognition · IEEE Transactions on Circuits & Systems II Express Briefs · · cited 107× · DOI ↗ · more like this ·

The softmax function has been widely used in deep neural networks (DNNs), and studies on efficient hardware accelerators for DNN have also attracted tremendous attention. However, it is very challenging to design efficient hardware architectures for softmax because of the expensive exponentiation and division calculations in it. In this brief, the softmax function is firstly simplified by exploring algorithmic strength reductions. Afterwards, a hardware-friendly and precision-adjustable calculation method for softmax is proposed, which can meet different precision requirements in various deep learning (DL) tasks. Based on the above innovations, an efficient architecture for softmax is presented. By tuning the parameter P , the system accuracy and complexity can be adjusted dynamically to achieve a good tradeoff between them. The proposed design is coded using hardware description…

Step Construction of Visual Cryptography Schemes

Feng Liu, Chuan-Kun Wu, Xi-Jun Lin

Computer Vision and Pattern Recognition · IEEE Transactions on Information Forensics and Security · · cited 92× · DOI ↗ · more like this ·

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> Two common drawbacks of the visual cryptography scheme (VCS) are the large pixel expansion of each share image and the small contrast of the recovered secret image. In this paper, we propose a <emphasis emphasistype="italic">step construction</emphasis> to construct <formula formulatype="inline"><tex Notation="TeX">$\hbox{VCS}_{\rm OR}$</tex></formula> and <formula formulatype="inline"><tex Notation="TeX">$\hbox{VCS}_{\rm XOR}$</tex></formula> for general access structure by applying (2,2)-VCS recursively, where a participant may receive multiple share images. The proposed step construction generates <formula formulatype="inline"><tex Notation="TeX">$\hbox{VCS}_{\rm OR}$</tex></formula> and <formula formulatype="inline"><tex Notation="TeX">$\hbox{VCS}_{\rm XOR}$</tex></formula> which have…

An efficient clustering method for k-anonymization

Jun-Lin Lin, Meng-Cheng Wei

Artificial Intelligence · · cited 92× · DOI ↗ · more like this ·

The k-anonymity model is a privacy-preserving approach that has been extensively studied for the past few years. To minimize the information loss due to anonymization, it is crucial to group similar data together and then anonymize each group individually. This work proposes a clustering-based k-anonymization method that runs in O(n2/k) time. We experimentally compare our method with another clustering-based k-anonymization method recently proposed by Byun et al. Even though their method has a time complexity of O(n2), the experiments show that our method outperforms their method with respect to information loss and resilience to outliers.

Accelerating Recurrent Neural Networks: A Memory-Efficient Approach

Zhisheng Wang, Jun Lin, Zhongfeng Wang

Artificial Intelligence · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · · cited 88× · DOI ↗ · more like this ·

Recurrent neural networks (RNNs) have achieved the state-of-the-art performance on various sequence learning tasks due to their powerful sequence modeling capability. However, RNNs usually require a large number of parameters and high computational complexity. Hence, it is quite challenging to implement complex RNNs on embedded devices with stringent memory and latency requirement. In this paper, we first present a novel hybrid compression method for a widely used RNN variant, long-short term memory (LSTM), to tackle these implementation challenges. By properly using circulant matrices, forward nonlinear function approximation, and efficient quantization schemes with a retrain-based training strategy, the proposed compression method can reduce more than 95% of memory usage with negligible accuracy loss when verified under language modeling and speech recognition tasks. An efficient…

An Energy-Efficient Architecture for Binary Weight Convolutional Neural Networks

Yizhi Wang, Jun Lin, Zhongfeng Wang

Computer Vision and Pattern Recognition · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · · cited 87× · DOI ↗ · more like this ·

Binary weight convolutional neural networks (BCNNs) can achieve near state-of-the-art classification accuracy and have far less computation complexity compared with traditional CNNs using high-precision weights. Due to their binary weights, BCNNs are well suited for vision-based Internet-of-Things systems being sensitive to power consumption. BCNNs make it possible to achieve very high throughput with moderate power dissipation. In this paper, an energy-efficient architecture for BCNNs is proposed. It fully exploits the binary weights and other hardware-friendly characteristics of BCNNs. A judicious processing schedule is proposed so that off-chip I/O access is minimized and activations are maximally reused. To significantly reduce the critical path delay, we introduce optimized compressor trees and approximate binary multipliers with two novel compensation schemes. The latter is able…

Mining association rules: anti-skew algorithms

Jun-Lin Lin, Margaret H. Dunham

Information Systems · · cited 82× · DOI ↗ · more like this ·

Mining association rules among items in a large database has been recognized as one of the most important data mining problems. All proposed approaches for this problem require scanning the entire database at least or almost twice in the worst case. We propose several techniques which overcome the problem of data skew in the basket data. These techniques reduce the maximum number of scans to less than 2, and in most cases find all association rules in about 1 scan. Our algorithms employ prior knowledge collected during the mining process and/or via sampling, to further reduce the number of candidate itemsets and identify false candidate itemsets at an earlier stage.

Poirot: A Distributed Tool Supporting Enterprise-Wide Automated Traceability

Jun Lin, Chan Chou Lin, J. C. Huang +7 more

Information Systems · · cited 81× · DOI ↗ · more like this ·

Poirot is a Web-based tool supporting traceability of distributed heterogeneous software artifacts. A probabilistic network model is used to generate traces between requirements, design elements, code and other artifacts stored in distributed 3 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">rd</sup> party case tools such as DOORS, rational rose, and source code repositories. The tool is designed with extensibility in mind, so that additional artifact types and 3 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">rd</sup> party case tools can be easily added. Trace results are displayed in both a textual and visual format. This paper briefly describes the underlying probabilistic model, and the user interface of the tool, and then discusses Poirot's deployment and use in an industrial setting

Containing the Spread of Coronavirus Disease 2019 (COVID-19): Meteorological Factors and Control Strategies

Jun Lin, Weihao Huang, Muchen Wen +9 more

Modeling and Simulation · medRxiv · · cited 79× · DOI ↗ · more like this ·

Abstract The novel coronavirus disease 2019 (COVID-19) has spread globally and the meteorological factors vary greatly across the world. Understanding the effect of meteorological factors and control strategies on COVID-19 transmission is critical to contain the epidemic. Using individual-level data in mainland China, Hong Kong, and Singapore, and the number of confirmed cases in other regions, we explore the effect of temperature, relative humidity, and control measures on the spread of COVID-19. We found that high temperature mitigates the transmission of the disease. High relative humidity promotes COVID-19 transmission when temperature is low, but tends to reduce transmission when temperature is high. Implementing classical control measures can dramatically slow the spread of the disease. However, due to the occurrence of pre-symptomatic infections, the effect of the measures to…

An Efficient List Decoder Architecture for Polar Codes

Jun Lin, Zhiyuan Yan

cs.AR · · cited 76× · arXiv ↗ · more like this ·

Long polar codes can achieve the symmetric capacity of arbitrary binary-input discrete memoryless channels under a low complexity successive cancelation (SC) decoding algorithm. However, for polar codes with short and moderate code length, the decoding performance of the SC algorithm is inferior. The cyclic redundancy check (CRC) aided successive cancelation list (SCL) decoding algorithm has better error performance than the SC algorithm for short or moderate polar codes. In this paper, we propose an efficient list decoder architecture for the CRC aided SCL algorithm, based on both algorithmic reformulations and architectural techniques. In particular, an area efficient message memory architecture is proposed to reduce the area of the proposed decoder architecture. An efficient path pruning unit suitable for large list size is also proposed. For a polar code of length 1024 and rate…

Object Segmentation of Database Images by Dual Multiscale Morphological Reconstructions and Retrieval Applications

Jiann‐Jone Chen, Chun-Rong Su, W. Eric L. Grimson +2 more

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · · cited 74× · DOI ↗ · more like this ·

Processing images for specific targets on a large scale has to handle various kinds of contents with regular processing steps. To segment objects in one image, we utilized dual multiScalE Graylevel mOrphological open and close recoNstructions (SEGON) to build a background (BG) gray-level variation mesh, which can help to identify BG and object regions. It was developed from a macroscopic perspective on image BG gray levels and implemented using standard procedures, thus robustly dealing with large-scale database images. The image segmentation capability of existing methods can be exploited by the BG mesh to improve object segmentation accuracy. To evaluate the segmentation accuracy, the probability of coherent segmentation labeling, i.e., the normalized probability random index (PRI), between a computer-segmented image and the hand-labeled one is computed for comparisons. Content-based…

E-LSTM: An Efficient Hardware Architecture for Long Short-Term Memory

Meiqi Wang, Zhisheng Wang, Jinming Lu +2 more

Artificial Intelligence · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · · cited 73× · DOI ↗ · more like this ·

Long Short-Term Memory (LSTM) and its variants have been widely adopted in many sequential learning tasks, such as speech recognition and machine translation. Significant accuracy improvements can be achieved using complex LSTM model with a large memory requirement and high computational complexity, which is time-consuming and energy demanding. The low-latency and energy-efficiency requirements of the real-world applications make model compression and hardware acceleration for LSTM an urgent need. In this paper, several hardware-efficient network compression schemes are introduced first, including structured top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> pruning, clipped gating, and multiplication-free quantization, to reduce the model size and the number of…

Efficient Decoder Design for Nonbinary Quasicyclic LDPC Codes

Jun Lin, Jin Sha, Zhongfeng Wang +1 more

Computer Networks and Communications · IEEE Transactions on Circuits and Systems I Regular Papers · · cited 70× · DOI ↗ · more like this ·

This paper addresses decoder design for nonbinary quasicyclic low-density parity-check (QC-LDPC) codes. First, a novel decoding algorithm is proposed to eliminate the multiplications over Galois field for check node processing. Then, a partially parallel architecture for check node processing units and an optimized architecture for variable node processing units are developed based on the new decoding algorithm. Thereafter, an efficient decoder structure dedicated to a promising class of high-performance nonbinary QC-LDPC codes is presented for the first time. Moreover, an ASIC implementation for a (620, 310) nonbinary QC-LDPC code decoder over GF(32) is designed to demonstrate the efficiency of the presented techniques.

Evaluations on Deep Neural Networks Training Using Posit Number System

Jinming Lu, Chao Fang, Mingyang Xu +2 more

Computer Vision and Pattern Recognition · IEEE Transactions on Computers · · cited 68× · DOI ↗ · more like this ·

The training of Deep Neural Networks (DNNs) brings enormous memory requirements and computational complexity, which makes it a challenge to train DNN models on resource-constrained devices. Training DNNs with reduced-precision data representation is crucial to mitigate this problem. In this article, we conduct a thorough investigation on training DNNs with low-bit posit numbers, a Type-III universal number (Unum). Through a comprehensive analysis of quantization with various data formats, it is demonstrated that the posit format shows great potential to be employed in the training of DNNs. Moreover, a DNN training framework using 8-bit posit is proposed with a novel tensor-wise scaling scheme. The experiments show the same performance as the state-of-the-art (SOTA) across multiple datasets (MNIST, CIFAR-10, ImageNet, and Penn Treebank) and model architectures (LeNet-5, AlexNet, ResNet…

An Efficient and Flexible Accelerator Design for Sparse Convolutional Neural Networks

Xiaoru Xie, Jun Lin, Zhongfeng Wang +1 more

Computer Vision and Pattern Recognition · IEEE Transactions on Circuits and Systems I Regular Papers · · cited 64× · DOI ↗ · more like this ·

Designing hardware accelerators for convolutional neural networks (CNNs) has recently attracted tremendous attention. Plenty of existing accelerators are built for dense CNNs or structured sparse CNNs. By contrast, unstructured sparse CNNs can achieve higher compression ratio with equivalent accuracy. However, their corresponding hardware implementations generally suffer from load imbalance and conflict access to on-chip buffers, which results in under utilization of processing elements (PEs). To tackle these issues, we propose a hardware/power-efficient and highly flexible architecture to support both unstructured and structured sparse CNNs with various configurations. Firstly, we propose an efficient weight reordering algorithm to preprocess compressed weights and balance the workload of PEs. Secondly, an adaptive on-chip dataflow, namely hybrid parallel (HP) dataflow, is introduced…

Topic-Oriented Spoken Dialogue Summarization for Customer Service with Saliency-Aware Topic Modeling

Yicheng Zou, Lujun Zhao, Yangyang Kang +7 more

cs.CL · · cited 64× · arXiv ↗ · more like this ·

In a customer service system, dialogue summarization can boost service efficiency by automatically creating summaries for long spoken dialogues in which customers and agents try to address issues about specific topics. In this work, we focus on topic-oriented dialogue summarization, which generates highly abstractive summaries that preserve the main ideas from dialogues. In spoken dialogues, abundant dialogue noise and common semantics could obscure the underlying informative content, making the general topic modeling approaches difficult to apply. In addition, for customer service, role-specific information matters and is an indispensable part of a summary. To effectively perform topic modeling on dialogues and capture multi-role information, in this work we propose a novel topic-augmented two-stage dialogue summarizer (TDS) jointly with a saliency-aware neural topic model (SATM) for…

Symbol-Decision Successive Cancellation List Decoder for Polar Codes

Chenrong Xiong, Jun Lin, Zhiyuan Yan

cs.IT · · cited 64× · arXiv ↗ · more like this ·

Polar codes are of great interests because they provably achieve the capacity of both discrete and continuous memoryless channels while having an explicit construction. Most existing decoding algorithms of polar codes are based on bit-wise hard or soft decisions. In this paper, we propose symbol-decision successive cancellation (SC) and successive cancellation list (SCL) decoders for polar codes, which use symbol-wise hard or soft decisions for higher throughput or better error performance. First, we propose to use a recursive channel combination to calculate symbol-wise channel transition probabilities, which lead to symbol decisions. Our proposed recursive channel combination also has a lower complexity than simply combining bit-wise channel transition probabilities. The similarity between our proposed method and Arikan's channel transformations also helps to share hardware resources…

A High Throughput List Decoder Architecture for Polar Codes

Jun Lin, Chenrong Xiong, Zhiyuan Yan

cs.AR · · cited 55× · arXiv ↗ · more like this ·

While long polar codes can achieve the capacity of arbitrary binary-input discrete memoryless channels when decoded by a low complexity successive cancelation (SC) algorithm, the error performance of the SC algorithm is inferior for polar codes with finite block lengths. The cyclic redundancy check (CRC) aided successive cancelation list (SCL) decoding algorithm has better error performance than the SC algorithm. However, current CRC aided SCL (CA-SCL) decoders still suffer from long decoding latency and limited throughput. In this paper, a reduced latency list decoding (RLLD) algorithm for polar codes is proposed. Our RLLD algorithm performs the list decoding on a binary tree, whose leaves correspond to the bits of a polar code. In existing SCL decoding algorithms, all the nodes in the tree are traversed and all possibilities of the information bits are considered. Instead, our RLLD…

TIE

Chunhua Deng, Fangxuan Sun, Xuehai Qian +3 more

Computer Vision and Pattern Recognition · · cited 53× · DOI ↗ · more like this ·

In the era of artificial intelligence (AI), deep neural networks (DNNs) have emerged as the most important and powerful AI technique. However, large DNN models are both storage and computation intensive, posing significant challenges for adopting DNNs in resource-constrained scenarios. Thus, model compression becomes a crucial technique to ensure wide deployment of DNNs.

Realistic Interpretation of Quantum Mechanics and Encounter-Delayed-Choice Experiment

Gui-Lu Long, Wei Qin, Zhe Yang +1 more

quant-ph · · cited 52× · arXiv ↗ · more like this ·

A realistic interpretation(REIN) of wave function in quantum mechanics is briefly presented in this work. In REIN, the wave function of a microscopic object is just its real existence rather than a mere mathematical description. Quantum object can exist in disjoint regions of space which just as the wave function distributes, travels at a finite speed, and collapses instantly upon a measurement. The single photon interference in a Mach-Zehnder interferometer is analyzed using REIN. In particular, we proposed and experimentally implemented a generalized delayed-choice experiment, the encounter-delayed-choice(EDC) experiment, in which the second beam splitter is inserted at the encounter of the two sub-waves from the two arms. In the EDC experiment, the front parts of wave functions before the beam splitter insertion do not interfere and show the particle nature, and the back parts of the…

Flexible LDPC Decoder Design for Multigigabit-per-Second Applications

Chuan Zhang, Zhongfeng Wang, Jin Sha +2 more

Computer Networks and Communications · IEEE Transactions on Circuits and Systems I Regular Papers · · cited 52× · DOI ↗ · more like this ·

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> Low-density parity-check (LDPC) codes are one of the most promising error-correcting codes approaching Shannon capacity and have been adopted in many applications. However, the efficient implementation of high-throughput LDPC decoders adaptable for various channel conditions still remains challenging. In this paper, a low-complexity reconfigurable VLSI architecture for high-speed LDPC decoders is presented. Shift-LDPC codes are incorporated within the design and have shown not only comparable decoding performance to computer-generated random codes but also high hardware efficiency in high-speed applications. The single-minimum Min-Sum decoding scheme and the nonuniform quantization scheme are explored to reduce the complexity of computing core and the memory requirement. The well-known Benes…

DynExit: A Dynamic Early-Exit Strategy for Deep Residual Networks

Meiqi Wang, Jianqiao Mo, Jun Lin +2 more

Computer Vision and Pattern Recognition · · cited 46× · DOI ↗ · more like this ·

Early-exit is a kind of technique to terminate a pre-specified computation at an early stage depending on the input samples and has been introduced to reduce energy consumption for Deep Neural Networks (DNNs). Previous early-exit approaches suffered from the burden of manually tuning early-exit loss-weights to find a good trade-off between complexity reduction and system accuracy. In this work, we first propose DynExit, a dynamic loss-weight modification strategy for ResNets, which adaptively modifies the ratio of different exit branches and searches for a proper spot for both accuracy and cost. Then, an efficient hardware unit for early-exit branches is developed, which can be easily integrated to existing hardware architectures of DNNs to reduce average computing latency and energy cost. Experimental results show that the proposed DynExit strategy can reduce up to 43.6% FLOPS compared…