arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Xue Lin

337 papers · 2000–2026

2000: 22001: 32002: 42003: 22004: 92005: 62006: 112007: 122008: 112009: 102010: 82011: 132012: 62013: 112014: 112015: 52016: 132017: 82018: 202019: 212020: 392021: 242022: 192023: 132024: 262025: 192026: 11
2000peak 39 in 20202026

Frequent co-authors

Papers

showing 1–25 of 337 RSS

Topology Attack and Defense for Graph Neural Networks: An Optimization Perspective

Kaidi Xu, Hongge Chen, Sijia Liu +4 more

cs.LG · · cited 551× · arXiv ↗ · more like this ·

Graph neural networks (GNNs) which apply the deep neural networks to graph data have achieved significant performance for the task of semi-supervised node classification. However, only few work has addressed the adversarial robustness of GNNs. In this paper, we first present a novel gradient-based attack method that facilitates the difficulty of tackling discrete graph data. When comparing to current adversarial attacks on GNNs, the results show that by only perturbing a small number of edge perturbations, including addition and deletion, our optimization-based attack can lead to a noticeable decrease in classification performance. Moreover, leveraging our gradient-based attack, we propose the first optimization-based adversarial training for GNNs. Our method yields higher robustness against both different gradient based and greedy attack methods without sacrificing classification…

Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Complete and Incomplete Neural Network Robustness Verification

Shiqi Wang, Huan Zhang, Kaidi Xu +4 more

cs.LG · · cited 358× · arXiv ↗ · more like this ·

Bound propagation based incomplete neural network verifiers such as CROWN are very efficient and can significantly accelerate branch-and-bound (BaB) based complete verification of neural networks. However, bound propagation cannot fully handle the neuron split constraints introduced by BaB commonly handled by expensive linear programming (LP) solvers, leading to loose bounds and hurting verification efficiency. In this work, we develop $β$-CROWN, a new bound propagation based method that can fully encode neuron splits via optimizable parameters $β$ constructed from either primal or dual space. When jointly optimized in intermediate layers, $β$-CROWN generally produces better bounds than typical LP verifiers with neuron split constraints, while being as efficient and parallelizable as CROWN on GPUs. Applied to complete robustness verification benchmarks, $β$-CROWN with BaB is up to three…

Feature Distillation: DNN-Oriented JPEG Compression Against Adversarial Examples

Zihao Liu, Qi Liu, Tao Liu +4 more

cs.CV · · cited 345× · arXiv ↗ · more like this ·

Image compression-based approaches for defending against the adversarial-example attacks, which threaten the safety use of deep neural networks (DNN), have been investigated recently. However, prior works mainly rely on directly tuning parameters like compression rate, to blindly reduce image features, thereby lacking guarantee on both defense efficiency (i.e. accuracy of polluted images) and classification accuracy of benign images, after applying defense methods. To overcome these limitations, we propose a JPEG-based defensive compression framework, namely "feature distillation", to effectively rectify adversarial examples without impacting classification accuracy on benign data. Our framework significantly escalates the defense efficiency with marginal accuracy reduction using a two-step method: First, we maximize malicious features filtering of adversarial input perturbations by…

Fast and Complete: Enabling Complete Neural Network Verification with Rapid and Massively Parallel Incomplete Verifiers

Kaidi Xu, Huan Zhang, Shiqi Wang +4 more

cs.AI · · cited 298× · arXiv ↗ · more like this ·

Formal verification of neural networks (NNs) is a challenging and important problem. Existing efficient complete solvers typically require the branch-and-bound (BaB) process, which splits the problem domain into sub-domains and solves each sub-domain using faster but weaker incomplete verifiers, such as Linear Programming (LP) on linearly relaxed sub-domains. In this paper, we propose to use the backward mode linear relaxation based perturbation analysis (LiRPA) to replace LP during the BaB process, which can be efficiently implemented on the typical machine learning accelerators such as GPUs and TPUs. However, unlike LP, LiRPA when applied naively can produce much weaker bounds and even cannot check certain conflicts of sub-domains during splitting, making the entire procedure incomplete after BaB. To address these challenges, we apply a fast gradient based bound tightening procedure…

CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-CirculantWeight Matrices

Caiwen Ding, Siyu Liao, Yanzhi Wang +13 more

cs.CV · · cited 288× · arXiv ↗ · more like this ·

Large-scale deep neural networks (DNNs) are both compute and memory intensive. As the size of DNNs continues to grow, it is critical to improve the energy efficiency and performance while maintaining accuracy. For DNNs, the model size is an important factor affecting performance, scalability and energy efficiency. Weight pruning achieves good compression ratios but suffers from three drawbacks: 1) the irregular network structure after pruning; 2) the increased training complexity; and 3) the lack of rigorous guarantee of compression ratio and inference accuracy. To overcome these limitations, this paper proposes CirCNN, a principled approach to represent weights and process neural networks using block-circulant matrices. CirCNN utilizes the Fast Fourier Transform (FFT)-based fast multiplication, simultaneously reducing the computational complexity (both in inference and training) from…

PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning

Wei Niu, Xiaolong Ma, Sheng Lin +5 more

cs.LG · · cited 257× · arXiv ↗ · more like this ·

With the emergence of a spectrum of high-end mobile devices, many applications that formerly required desktop-level computation capability are being transferred to these devices. However, executing the inference of Deep Neural Networks (DNNs) is still challenging considering high computation and storage demands, specifically, if real-time performance with high accuracy is needed. Weight pruning of DNNs is proposed, but existing schemes represent two extremes in the design space: non-structured pruning is fine-grained, accurate, but not hardware friendly; structured pruning is coarse-grained, hardware-efficient, but with higher accuracy loss. In this paper, we introduce a new dimension, fine-grained pruning patterns inside the coarse-grained structures, revealing a previously unknown point in design space. With the higher accuracy enabled by fine-grained pruning patterns, the unique…

Task Scheduling with Dynamic Voltage and Frequency Scaling for Energy Minimization in the Mobile Cloud Computing Environment

Xue Lin, Yanzhi Wang, Qing Xie +1 more

Computer Networks and Communications · IEEE Transactions on Services Computing · · cited 256× · DOI ↗ · more like this ·

Mobile cloud computing (MCC) offers significant opportunities in performance enhancement and energy saving for mobile, battery-powered devices. Applications running on mobile devices may be represented by task graphs. This work investigates the problem of scheduling tasks (which belong to the same or possibly different applications) in the MCC environment. More precisely, the scheduling problem involves the following steps: (i) determining the tasks to be offloaded onto the cloud, (ii) mapping the remaining tasks onto (potentially heterogeneous) local cores in the mobile device, (iii) determining the frequencies for executing local tasks, and (iv) scheduling tasks on the cores (for in-house tasks) and the wireless communication channels (for offloaded tasks) such that the task-precedence requirements and the application completion time constraint are satisfied while the total energy…

Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness

Pu Zhao, Pin-Yu Chen, Payel Das +2 more

cs.LG · · cited 219× · arXiv ↗ · more like this ·

Mode connectivity provides novel geometric insights on analyzing loss landscapes and enables building high-accuracy pathways between well-trained neural networks. In this work, we propose to employ mode connectivity in loss landscapes to study the adversarial robustness of deep neural networks, and provide novel methods for improving this robustness. Our experiments cover various types of adversarial attacks applied to different network architectures and datasets. When network models are tampered with backdoor or error-injection attacks, our results demonstrate that the path connection learned using limited amount of bonafide data can effectively mitigate adversarial effects while maintaining the original accuracy on clean data. Therefore, mode connectivity provides users with the power to repair backdoored or error-injected models. We also use mode connectivity to investigate the loss…

PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-time Execution on Mobile Devices

Xiaolong Ma, Fu-Ming Guo, Wei Niu +5 more

cs.LG · · cited 209× · arXiv ↗ · more like this ·

Model compression techniques on Deep Neural Network (DNN) have been widely acknowledged as an effective way to achieve acceleration on a variety of platforms, and DNN weight pruning is a straightforward and effective method. There are currently two mainstreams of pruning methods representing two extremes of pruning regularity: non-structured, fine-grained pruning can achieve high sparsity and accuracy, but is not hardware friendly; structured, coarse-grained pruning exploits hardware-efficient structures in pruning, but suffers from accuracy drop when the pruning rate is high. In this paper, we introduce PCONV, comprising a new sparsity dimension, -- fine-grained pruning patterns inside the coarse-grained structures. PCONV comprises two types of sparsities, Sparse Convolution Patterns (SCP) which is generated from intra-convolution kernel pruning and connectivity sparsity generated from…

ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Method of Multipliers

Ao Ren, Tianyun Zhang, Shaokai Ye +5 more

cs.LG · · cited 176× · arXiv ↗ · more like this ·

To facilitate efficient embedded and hardware implementations of deep neural networks (DNNs), two important categories of DNN model compression techniques: weight pruning and weight quantization are investigated. The former leverages the redundancy in the number of weights, whereas the latter leverages the redundancy in bit representation of weights. However, there lacks a systematic framework of joint weight pruning and quantization of DNNs, thereby limiting the available model compression ratio. Moreover, the computation reduction, energy efficiency improvement, and hardware performance overhead need to be accounted for besides simply model size reduction. To address these limitations, we present ADMM-NN, the first algorithm-hardware co-optimization framework of DNNs using Alternating Direction Method of Multipliers (ADMM), a powerful technique to deal with non-convex optimization…

Structured Adversarial Attack: Towards General Implementation and Better Interpretability

Kaidi Xu, Sijia Liu, Pu Zhao +6 more

cs.LG · · cited 172× · arXiv ↗ · more like this ·

When generating adversarial examples to attack deep neural networks (DNNs), Lp norm of the added perturbation is usually used to measure the similarity between original image and adversarial example. However, such adversarial attacks perturbing the raw input spaces may fail to capture structural information hidden in the input. This work develops a more general attack model, i.e., the structured attack (StrAttack), which explores group sparsity in adversarial perturbations by sliding a mask through images aiming for extracting key spatial structures. An ADMM (alternating direction method of multipliers)-based framework is proposed that can split the original problem into a sequence of analytically solvable subproblems and can be generalized to implement other attacking methods. Strong group sparsity is achieved in adversarial perturbations even with the same level of Lp norm distortion…

Finite-Time Consensus of Switched Multiagent Systems

Xue Lin, Yuanshi Zheng

Computer Networks and Communications · IEEE Transactions on Systems Man and Cybernetics Systems · · cited 166× · DOI ↗ · more like this ·

This paper focuses on the finite-time consensus (FTC) problem of switched multiagent system (MAS) which is composed of continuous-time and discrete-time subsystems. Different from the existing results, each agent of this system is controlled by switching control method. To achieve consensus in finite time for the switched MAS, two types of consensus protocols (the FTC protocol and the fixed-time consensus (FdTC) protocol) are proposed. By using algebraic graph theory, Lyapunov theory and matrix theory, it is proved that the FTC problem in strongly connected network and leader-following network can be solved, respectively. When the initial states of agents are not available, the FdTC protocol is applied to solve the FTC problem. Simulations are provided to illustrate the effectiveness of our theoretical results.

Dirty Road Can Attack: Security of Deep Learning based Automated Lane Centering under Physical-World Attack

Takami Sato, Junjie Shen, Ningfei Wang +3 more

cs.CR · · cited 165× · arXiv ↗ · more like this ·

Automated Lane Centering (ALC) systems are convenient and widely deployed today, but also highly security and safety critical. In this work, we are the first to systematically study the security of state-of-the-art deep learning based ALC systems in their designed operational domains under physical-world adversarial attacks. We formulate the problem with a safety-critical attack goal, and a novel and domain-specific attack vector: dirty road patches. To systematically generate the attack, we adopt an optimization-based approach and overcome domain-specific design challenges such as camera frame inter-dependencies due to attack-influenced vehicle control, and the lack of objective function design for lane detection models. We evaluate our attack on a production ALC using 80 scenarios from real-world driving traces. The results show that our attack is highly effective with over 97.5%…

ZO-AdaMM: Zeroth-Order Adaptive Momentum Method for Black-Box Optimization

Xiangyi Chen, Sijia Liu, Kaidi Xu +4 more

cs.LG · · cited 158× · arXiv ↗ · more like this ·

The adaptive momentum method (AdaMM), which uses past gradients to update descent directions and learning rates simultaneously, has become one of the most popular first-order optimization methods for solving machine learning problems. However, AdaMM is not suited for solving black-box optimization problems, where explicit gradient forms are difficult or infeasible to obtain. In this paper, we propose a zeroth-order AdaMM (ZO-AdaMM) algorithm, that generalizes AdaMM to the gradient-free regime. We show that the convergence rate of ZO-AdaMM for both convex and nonconvex optimization is roughly a factor of $O(\sqrt{d})$ worse than that of the first-order AdaMM algorithm, where $d$ is problem size. In particular, we provide a deep understanding on why Mahalanobis distance matters in convergence of ZO-AdaMM and other AdaMM-type methods. As a byproduct, our analysis makes the first step…

Non-Structured DNN Weight Pruning -- Is It Beneficial in Any Platform?

Xiaolong Ma, Sheng Lin, Shaokai Ye +10 more

cs.LG · · cited 124× · arXiv ↗ · more like this ·

Large deep neural network (DNN) models pose the key challenge to energy efficiency due to the significantly higher energy consumption of off-chip DRAM accesses than arithmetic or SRAM operations. It motivates the intensive research on model compression with two main approaches. Weight pruning leverages the redundancy in the number of weights and can be performed in a non-structured, which has higher flexibility and pruning rate but incurs index accesses due to irregular weights, or structured manner, which preserves the full matrix structure with lower pruning rate. Weight quantization leverages the redundancy in the number of bits in weights. Compared to pruning, quantization is much more hardware-friendly, and has become a "must-do" step for FPGA and ASIC implementations. This paper provides a definitive answer to the question for the first time. First, we build ADMM-NN-S by extending…

MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge

Geng Yuan, Xiaolong Ma, Wei Niu +13 more

cs.LG · · cited 123× · arXiv ↗ · more like this ·

Recently, a new trend of exploring sparsity for accelerating neural network training has emerged, embracing the paradigm of training on the edge. This paper proposes a novel Memory-Economic Sparse Training (MEST) framework targeting for accurate and fast execution on edge devices. The proposed MEST framework consists of enhancements by Elastic Mutation (EM) and Soft Memory Bound (&S) that ensure superior accuracy at high sparsity ratios. Different from the existing works for sparse training, this current work reveals the importance of sparsity schemes on the performance of sparse training in terms of accuracy as well as training speed on real edge devices. On top of that, the paper proposes to employ data efficiency for further acceleration of sparse training. Our results suggest that unforgettable examples can be identified in-situ even during the dynamic exploration of sparsity masks…

Mix and Match: A Novel FPGA-Centric Deep Neural Network Quantization Framework

Sung-En Chang, Yanyu Li, Mengshu Sun +5 more

cs.LG · · cited 122× · arXiv ↗ · more like this ·

Deep Neural Networks (DNNs) have achieved extraordinary performance in various application domains. To support diverse DNN models, efficient implementations of DNN inference on edge-computing platforms, e.g., ASICs, FPGAs, and embedded systems, are extensively investigated. Due to the huge model size and computation amount, model compression is a critical step to deploy DNN models on edge devices. This paper focuses on weight quantization, a hardware-friendly model compression approach that is complementary to weight pruning. Unlike existing methods that use the same quantization scheme for all weights, we propose the first solution that applies different quantization schemes for different rows of the weight matrix. It is motivated by (1) the distribution of the weights in the different rows are not the same; and (2) the potential of achieving better utilization of heterogeneous FPGA…

Role of Stressful Life Events, Avoidant Coping Styles, and Neuroticism in Online Game Addiction among College Students: A Moderated Mediation Model

Huanhuan Li, Ying-min Zou, Jiaqi Wang +1 more

Sociology and Political Science · Frontiers in Psychology · · cited 98× · DOI ↗ · more like this ·

Online game addiction (OGA) is becoming a significant problem worldwide. The aim of this study was to explore the incidence of OGA and the roles of stressful life events, avoidant coping styles (ACSs), and neuroticism in OGA. A total of 651 Chinese college students were selected by random cluster sampling. Subjects completed the Chinese version of Young's eight-item Internet Addiction Scale (CIAS), Online Game Cognition Addiction Scale (OGCAS), Revised Eysenck Personality Questionnaire Short Scale in Chinese (EPQ-RSC), Chinese College-student Stress Questionnaire, and Coping Style Questionnaire. Structural equation modeling (SEM) was used to explore the interactive effects of stressful life events, ACSs, and neuroticism on OGA. Of the 651 participants in the sample, 31 (4.8%) were identified as addicts. The incidence of OGA was two times higher for males than females. The addicts had…

Auto-ViT-Acc: An FPGA-Aware Automatic Acceleration Framework for Vision Transformer with Mixed-Scheme Quantization

Zhengang Li, Mengshu Sun, Alec Lu +9 more

cs.CV · · cited 90× · arXiv ↗ · more like this ·

Vision transformers (ViTs) are emerging with significantly improved accuracy in computer vision tasks. However, their complex architecture and enormous computation/storage demand impose urgent needs for new hardware accelerator design methodology. This work proposes an FPGA-aware automatic ViT acceleration framework based on the proposed mixed-scheme quantization. To the best of our knowledge, this is the first FPGA-based ViT acceleration framework exploring model quantization. Compared with state-of-the-art ViT quantization work (algorithmic approach only without hardware acceleration), our quantization achieves 0.47% to 1.36% higher Top-1 accuracy under the same bit-width. Compared with the 32-bit floating-point baseline FPGA accelerator, our accelerator achieves around 5.6x improvement on the frame rate (i.e., 56.8 FPS vs. 10.0 FPS) with 0.71% accuracy drop on ImageNet dataset for…

FinCACTI: Architectural Analysis and Modeling of Caches with Deeply-Scaled FinFET Devices

Alireza Shafaei, Yanzhi Wang, Xue Lin +1 more

Hardware and Architecture · · cited 86× · DOI ↗ · more like this ·

This paper presents FinCACTI, a cache modeling tool based on CACTI which also supports deeply-scaled FinFET devices as well as more robust SRAM cells. In particular, FinFET devices optimized using advanced device simulators for 7nm process serve as the case study of the paper. Based on this 7nm FinFET process, characteristics of 6T and 8T SRAMs are calculated, and comparison results show that under the same stability requirements the 8T cell has smaller area and leakage power. SRAM and technological parameters of the 7nm FinFET are then incorporated into FinCACTI. According to architecture-level simulations, the 8T SRAM is suggested as the choice of memory cell for 7nm FinFET. Moreover, a 4MB cache in 7nm FinFET compared with 22nm (32nm) CMOS under same access latencies achieves 5x (9x) and 11x (24x) reduction in read energy and area, respectively.

FILM-QNN: Efficient FPGA Acceleration of Deep Neural Networks with Intra-Layer, Mixed-Precision Quantization

Mengshu Sun, Zhengang Li, Alec Lu +5 more

Computer Vision and Pattern Recognition · · cited 85× · DOI ↗ · more like this ·

With the trend to deploy Deep Neural Network (DNN) inference models on edge devices with limited resources, quantization techniques have been widely used to reduce on-chip storage and improve computation throughput. However, existing DNN quantization work deploying quantization below 8-bit may be either suffering from evident accuracy loss or facing a big gap between the theoretical improvement of computation throughput and the practical inference speedup. In this work, we propose a general framework, called FILM-QNN, to quantize and accelerate multiple DNN models across different embedded FPGA devices. First, we propose the novel intra-layer, mixed-precision quantization algorithm that assigns different precisions onto the filters of each layer. The candidate precision levels and assignment granularity are determined from our empirical study with the capability of preserving accuracy…

A Nested Two Stage Game-Based Optimization Framework in Mobile Cloud Computing System

Yanzhi Wang, Xue Lin, Massoud Pedram

Information Systems · · cited 83× · DOI ↗ · more like this ·

The rapidly developing cloud computing and virtualization techniques provide mobile devices with battery energy saving opportunities by allowing them to offload computation and execute applications remotely. A mobile device should judiciously decide whether to offload computation and which portion of application should be offloaded to the cloud. In this paper, we consider a mobile cloud computing (MCC) interaction system consisting of multiple mobile devices and the cloud computing facilities. We provide a nested two stage game formulation for the MCC interaction system. In the first stage, each mobile device determines the portion of its service requests for remote processing in the cloud. In the second stage, the cloud computing facilities allocate a portion of its total resources for service request processing depending on the request arrival rate from all the mobile devices. The…

Fault Sneaking Attack: a Stealthy Framework for Misleading Deep Neural Networks

Pu Zhao, Siyue Wang, Cheng Gongye +3 more

cs.LG · · cited 82× · arXiv ↗ · more like this ·

Despite the great achievements of deep neural networks (DNNs), the vulnerability of state-of-the-art DNNs raises security concerns of DNNs in many application domains requiring high reliability.We propose the fault sneaking attack on DNNs, where the adversary aims to misclassify certain input images into any target labels by modifying the DNN parameters. We apply ADMM (alternating direction method of multipliers) for solving the optimization problem of the fault sneaking attack with two constraints: 1) the classification of the other images should be unchanged and 2) the parameter modifications should be minimized. Specifically, the first constraint requires us not only to inject designated faults (misclassifications), but also to hide the faults for stealthy or sneaking considerations by maintaining model accuracy. The second constraint requires us to minimize the parameter…

Defensive Dropout for Hardening Deep Neural Networks under Adversarial Attacks

Siyue Wang, Xiao Wang, Pu Zhao +4 more

cs.CR · · cited 76× · arXiv ↗ · more like this ·

Deep neural networks (DNNs) are known vulnerable to adversarial attacks. That is, adversarial examples, obtained by adding delicately crafted distortions onto original legal inputs, can mislead a DNN to classify them as any target labels. This work provides a solution to hardening DNNs under adversarial attacks through defensive dropout. Besides using dropout during training for the best test accuracy, we propose to use dropout also at test time to achieve strong defense effects. We consider the problem of building robust DNNs as an attacker-defender two-player game, where the attacker and the defender know each others' strategies and try to optimize their own strategies towards an equilibrium. Based on the observations of the effect of test dropout rate on test accuracy and attack success rate, we propose a defensive dropout algorithm to determine an optimal test dropout rate given the…

Reweighted Proximal Pruning for Large-Scale Language Representation

Fu-Ming Guo, Sijia Liu, Finlay S. Mungall +2 more

cs.LG · · cited 74× · arXiv ↗ · more like this ·

Recently, pre-trained language representation flourishes as the mainstay of the natural language understanding community, e.g., BERT. These pre-trained language representations can create state-of-the-art results on a wide range of downstream tasks. Along with continuous significant performance improvement, the size and complexity of these pre-trained neural models continue to increase rapidly. Is it possible to compress these large-scale language representation models? How will the pruned language representation affect the downstream multi-task transfer learning objectives? In this paper, we propose Reweighted Proximal Pruning (RPP), a new pruning method specifically designed for a large-scale language representation model. Through experiments on SQuAD and the GLUE benchmark suite, we show that proximal pruned BERT keeps high accuracy for both the pre-training task and the downstream…