arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Fuxun Yu

71 papers · 2018–2026

2018: 122019: 132020: 82021: 62022: 92023: 62024: 72025: 72026: 3
2018peak 13 in 20192026

Frequent co-authors

Papers

showing 1–25 of 71 RSS

How convolutional neural networks see the world --- A survey of convolutional neural network visualization methods

Zhuwei Qin, Fuxun Yu, Chenchen Liu +1 more

Artificial Intelligence · Mathematical Foundations of Computing · · cited 206× · DOI ↗ · more like this ·

Nowadays, the Convolutional Neural Networks (CNNs) have achieved impressive performance on many computer vision related tasks, such as object detection, image recognition, image retrieval, etc. These achievements benefit from the CNNs' outstanding capability to learn the input features with deep layers of neuron structures and iterative training process. However, these learned features are hard to identify and interpret from a human vision perspective, causing a lack of understanding of the CNNs' internal working mechanism. To improve the CNN interpretability, the CNN visualization is well utilized as a qualitative analysis method, which translates the internal features into visually perceptible patterns. And many CNN visualization works have been proposed in the literature to interpret the CNN in perspectives of network structure, operation, and semantic concept.In this paper, we…

Helios: Heterogeneity-Aware Federated Learning with Dynamically Balanced Collaboration

Zirui Xu, Fuxun Yu, Jinjun Xiong +1 more

cs.DC · · cited 68× · arXiv ↗ · more like this ·

In this paper, we propose Helios, a heterogeneity-aware FL framework to tackle the straggler issue. Helios identifies individual devices' heterogeneous training capability, and therefore the expected neural network model training volumes regarding the collaborative training pace. For straggling devices, a "soft-training" method is proposed to dynamically compress the original identical training model into the expected volume through a rotating neuron training approach. With extensive algorithm analysis and optimization schemes, the stragglers can be accelerated while retaining the convergence for local training as well as federated collaboration.

Automated Runtime-Aware Scheduling for Multi-Tenant DNN Inference on GPU

Fuxun Yu, Shawn Bray, Di Wang +4 more

cs.DC · · cited 61× · arXiv ↗ · more like this ·

With the fast development of deep neural networks (DNNs), many real-world applications are adopting multiple models to conduct compound tasks, such as co-running classification, detection, and segmentation models on autonomous vehicles. Such multi-tenant DNN inference cases greatly exacerbate the computational complexity and call for comprehensive collaboration for graph-level operator scheduling, runtime-level resource awareness, as well as hardware scheduler support. However, the current scheduling support for such multi-tenant inference is still relatively backward. In this work, we propose a resource-aware scheduling framework for efficient multi-tenant DNN inference on GPU, which automatically coordinates DNN computing in different execution levels. Leveraging the unified scheduling intermediate representation and the automated ML-based searching algorithm, optimal schedules could…

Interpreting and Evaluating Neural Network Robustness

Fuxun Yu, Zhuwei Qin, Chenchen Liu +3 more

cs.LG · · cited 60× · arXiv ↗ · more like this ·

Recently, adversarial deception becomes one of the most considerable threats to deep neural networks. However, compared to extensive research in new designs of various adversarial attacks and defenses, the neural networks' intrinsic robustness property is still lack of thorough investigation. This work aims to qualitatively interpret the adversarial attack and defense mechanism through loss visualization, and establish a quantitative metric to evaluate the neural network model's intrinsic robustness. The proposed robustness metric identifies the upper bound of a model's prediction divergence in the given domain and thus indicates whether the model can maintain a stable prediction. With extensive experiments, our metric demonstrates several advantages over conventional adversarial testing accuracy based robustness estimation: (1) it provides a uniformed evaluation to models with…

Fed2

Fuxun Yu, Weishan Zhang, Zhuwei Qin +5 more

Artificial Intelligence · · cited 57× · DOI ↗ · more like this ·

Federated learning learns from scattered data by fusing collaborative models from local nodes. However, conventional coordinate-based model averaging by FedAvg ignored the random information encoded per parameter and may suffer from structural feature misalignment. In this work, we propose Fed2, a feature-aligned federated learning framework to resolve this issue by establishing a firm structure-feature alignment across the collaborative models. Fed2 is composed of two major designs: First, we design a feature-oriented model structure adaptation method to ensure explicit feature allocation in different neural network structures. Applying the structure adaptation to collaborative models, matchable structures with similar feature information can be initialized at the very early training stage. During the federated learning process, we then propose a feature paired averaging scheme to…

How convolutional neural network see the world - A survey of convolutional neural network visualization methods

Zhuwei Qin, Fuxun Yu, Chenchen Liu +1 more

cs.CV · · cited 48× · arXiv ↗ · more like this ·

Nowadays, the Convolutional Neural Networks (CNNs) have achieved impressive performance on many computer vision related tasks, such as object detection, image recognition, image retrieval, etc. These achievements benefit from the CNNs outstanding capability to learn the input features with deep layers of neuron structures and iterative training process. However, these learned features are hard to identify and interpret from a human vision perspective, causing a lack of understanding of the CNNs internal working mechanism. To improve the CNN interpretability, the CNN visualization is well utilized as a qualitative analysis method, which translates the internal features into visually perceptible patterns. And many CNN visualization works have been proposed in the literature to interpret the CNN in perspectives of network structure, operation, and semantic concept. In this paper, we expect…

SC-UDA: Style and Content Gaps aware Unsupervised Domain Adaptation for Object Detection

Fuxun Yu, Di Wang, Yinpeng Chen +7 more

Artificial Intelligence · 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · · cited 46× · DOI ↗ · more like this ·

Current state-of-the-art object detectors can have significant performance drop when deployed in the wild due to domain gaps with training data. Unsupervised Domain Adaptation (UDA) is a promising approach to adapt detectors for new domains/environments without any expensive label cost. Previous mainstream UDA works for object detection usually focused on image-level and/or feature-level adaptation by using adversarial learning methods. In this work, we show that such adversarial-based methods can only reduce domain style gap, but cannot address the domain content gap that is also important for object detectors. To overcome this limitation, we propose the SC-UDA framework to concurrently reduce both gaps: We propose fine-grained domain style transfer to reduce the style gaps with finer image details preserved for detecting small objects; Then we leverage the pseudo label-based…

Progressive Weight Pruning of Deep Neural Networks using ADMM

Shaokai Ye, Tianyun Zhang, Kaiqi Zhang +10 more

cs.LG · · cited 42× · arXiv ↗ · more like this ·

Deep neural networks (DNNs) although achieving human-level performance in many domains, have very large model size that hinders their broader applications on edge computing devices. Extensive research work have been conducted on DNN model compression or pruning. However, most of the previous work took heuristic approaches. This work proposes a progressive weight pruning approach based on ADMM (Alternating Direction Method of Multipliers), a powerful technique to deal with non-convex optimization problems with potentially combinatorial constraints. Motivated by dynamic programming, the proposed method reaches extremely high pruning rate by using partial prunings with moderate pruning rates. Therefore, it resolves the accuracy degradation and long convergence time problems when pursuing extremely high pruning ratios. It achieves up to 34 times pruning rate for ImageNet dataset and 167…

Unsupervised Domain Adaptation for Object Detection via Cross-Domain Semi-Supervised Learning

Fuxun Yu, Di Wang, Yinpeng Chen +7 more

cs.CV · · cited 40× · arXiv ↗ · more like this ·

Current state-of-the-art object detectors can have significant performance drop when deployed in the wild due to domain gaps with training data. Unsupervised Domain Adaptation (UDA) is a promising approach to adapt models for new domains/environments without any expensive label cost. However, without ground truth labels, most prior works on UDA for object detection tasks can only perform coarse image-level and/or feature-level adaptation by using adversarial learning methods. In this work, we show that such adversarial-based methods can only reduce the domain style gap, but cannot address the domain content distribution gap that is shown to be important for object detectors. To overcome this limitation, we propose the Cross-Domain Semi-Supervised Learning (CDSSL) framework by leveraging high-quality pseudo labels to learn better representations from the target domain directly. To enable…

QuadraLib: A Performant Quadratic Neural Network Library for Architecture Optimization and Design Exploration

Zirui Xu, Fuxun Yu, Jinjun Xiong +1 more

cs.LG · · cited 36× · arXiv ↗ · more like this ·

The significant success of Deep Neural Networks (DNNs) is highly promoted by the multiple sophisticated DNN libraries. On the contrary, although some work have proved that Quadratic Deep Neuron Networks (QDNNs) show better non-linearity and learning capability than the first-order DNNs, their neuron design suffers certain drawbacks from theoretical performance to practical deployment. In this paper, we first proposed a new QDNN neuron architecture design, and further developed QuadraLib, a QDNN library to provide architecture optimization and design exploration for QDNNs. Extensive experiments show that our design has good performance regarding prediction accuracy and computation consumption on multiple learning tasks.

REIN the RobuTS: Robust DNN-Based Image Recognition in Autonomous Driving Systems

Fuxun Yu, Zhuwei Qin, Chenchen Liu +2 more

Computer Vision and Pattern Recognition · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · · cited 33× · DOI ↗ · more like this ·

In recent years, the neural network (NN) has shown its great potential in image recognition tasks of autonomous driving systems, such as traffic sign recognition, pedestrian detection, etc. However, theoretically well-trained NNs usually fail their performance when facing real-world scenarios. For example, adverse real-world conditions, e.g., bad weather and lighting conditions, can introduce different physical variations and cause considerable accuracy degradation. As for now, the generalization capability of NNs is still one of the most critical challenges for the autonomous driving system. To facilitate the robust image recognition tasks, in this work, we build the RobuTS dataset: a comprehensive Robust Traffic Sign Recognition dataset, which includes images with different environmental variations, e.g., rain, fog, darkening, and blurring. Then to enhance the NN's generalization…

Heterogeneous Federated Learning

Fuxun Yu, Weishan Zhang, Zhuwei Qin +5 more

cs.LG · · cited 32× · arXiv ↗ · more like this ·

Federated learning learns from scattered data by fusing collaborative models from local nodes. However, due to chaotic information distribution, the model fusion may suffer from structural misalignment with regard to unmatched parameters. In this work, we propose a novel federated learning framework to resolve this issue by establishing a firm structure-information alignment across collaborative models. Specifically, we design a feature-oriented regulation method ({$Ψ$-Net}) to ensure explicit feature information allocation in different neural network structures. Applying this regulating method to collaborative models, matchable structures with similar feature information can be initialized at the very early training stage. During the federated learning process under either IID or non-IID scenarios, dedicated collaboration schemes further guarantee ordered information distribution with…

Interpreting Adversarial Robustness: A View from Decision Surface in Input Space

Fuxun Yu, Chenchen Liu, Yanzhi Wang +2 more

cs.LG · · cited 32× · arXiv ↗ · more like this ·

One popular hypothesis of neural network generalization is that the flat local minima of loss surface in parameter space leads to good generalization. However, we demonstrate that loss surface in parameter space has no obvious relationship with generalization, especially under adversarial settings. Through visualizing decision surfaces in both parameter space and input space, we instead show that the geometry property of decision surface in input space correlates well with the adversarial robustness. We then propose an adversarial robustness indicator, which can evaluate a neural network's intrinsic robustness property without testing its accuracy under adversarial attacks. Guided by it, we further propose our robust training method. Without involving adversarial training, our method could enhance network's intrinsic adversarial robustness against various adversarial attacks.

ReForm

Zirui Xu, Fuxun Yu, Chenchen Liu +1 more

Computer Vision and Pattern Recognition · · cited 31× · DOI ↗ · more like this ·

Although the Deep Neural Network (DNN) technique has been widely applied in various applications, the DNN-based applications are still too computationally intensive for the resource-constrained mobile devices. Many works have been proposed to optimize the DNN computation performance, but most of them are limited in an algorithmic perspective, ignoring certain computing issues in practical deployment. To achieve the comprehensive DNN performance enhancement in practice, the expected DNN optimization works should closely cooperate with specific hardware and system constraints (i.e. computation capacity, energy cost, memory occupancy, and inference latency). Therefore, in this work, we propose ReForm -- a resource-aware DNN optimization framework. Through thorough mobile DNN computing analysis and innovative model reconfiguration schemes (i.e. ADMM based static model fine-tuning…

Supporting Massive DLRM Inference Through Software Defined Memory

Ehsan K. Ardestani, Changkyu Kim, Seung Jae Lee +17 more

cs.AR · · cited 29× · arXiv ↗ · more like this ·

Deep Learning Recommendation Models (DLRM) are widespread, account for a considerable data center footprint, and grow by more than 1.5x per year. With model size soon to be in terabytes range, leveraging Storage ClassMemory (SCM) for inference enables lower power consumption and cost. This paper evaluates the major challenges in extending the memory hierarchy to SCM for DLRM, and presents different techniques to improve performance through a Software Defined Memory. We show how underlying technologies such as Nand Flash and 3DXP differentiate, and relate to real world scenarios, enabling from 5% to 29% power savings.

LanCe: A Comprehensive and Lightweight CNN Defense Methodology against Physical Adversarial Attacks on Embedded Multimedia Applications

Zirui Xu, Fuxun Yu, Xiang Chen

cs.CV · · cited 29× · arXiv ↗ · more like this ·

Recently, adversarial attacks can be applied to the physical world, causing practical issues to various Convolutional Neural Networks (CNNs) powered applications. Most existing physical adversarial attack defense works only focus on eliminating explicit perturbation patterns from inputs, ignoring interpretation to CNN's intrinsic vulnerability. Therefore, they lack the expected versatility to different attacks and thereby depend on considerable data processing costs. In this paper, we propose LanCe -- a comprehensive and lightweight CNN defense methodology against different physical adversarial attacks. By interpreting CNN's vulnerability, we find that non-semantic adversarial perturbations can activate CNN with significantly abnormal activations and even overwhelm other semantic input patterns' activations. We improve the CNN recognition process by adding a self-verification stage to…

Distilling Critical Paths in Convolutional Neural Networks

Fuxun Yu, Zhuwei Qin, Xiang Chen

cs.CV · · cited 24× · arXiv ↗ · more like this ·

Neural network compression and acceleration are widely demanded currently due to the resource constraints on most deployment targets. In this paper, through analyzing the filter activation, gradients, and visualizing the filters' functionality in convolutional neural networks, we show that the filters in higher layers learn extremely task-specific features, which are exclusive for only a small subset of the overall tasks, or even a single class. Based on such findings, we reveal the critical paths of information flow for different classes. And by their intrinsic property of exclusiveness, we propose a critical path distillation method, which can effectively customize the convolutional neural networks to small ones with much smaller model size and less computation.

DiReCt

Zirui Xu, Zhuwei Qin, Fuxun Yu +2 more

Computer Vision and Pattern Recognition · Proceedings of the International Symposium on Low Power Electronics and Design · · cited 12× · DOI ↗ · more like this ·

Although Convolutional Neural Networks (CNNs) have been widely applied in various applications, their deployment in resource-constrained mobile systems remains a significant concern. To overcome the computation resource constraints, such as limited memory and energy capacity, many works are proposed for mobile CNN optimization. However, most of them lack a comprehensive modeling analysis of the CNN computation consumption and merely focus on static optimization schemes regardless of different mobile computation scenarios. In this work, we proposed DiReCt -- a resource-aware CNN reconfiguration system. Leveraging accurate CNN computation consumption modeling and mobile resource constraint analysis, DiReCt can reconfigure a CNN with different accuracy and resource consumption levels to adapt to various mobile computation scenarios. The experiment results show that: the proposed…

CAPTOR

Zhuwei Qin, Fuxun Yu, Chenchen Liu +1 more

Computer Vision and Pattern Recognition · · cited 11× · DOI ↗ · more like this ·

Nowadays, the evolution of deep learning and cloud service significantly promotes neural network based mobile applications. Although intelligent and prolific, those applications still lack certain flexibility: For classification tasks, neural networks are generally trained online with vast classification targets to cover various utilization contexts. However, only partial classes are practically tested due to individual mobile user preference and application specificity. Thus the unneeded classes cause considerable computation and communication cost. In this work, we propose CAPTOR - a class-level reconfiguration framework for Convolutional Neural Networks (CNNs). By identifying the class activation preference of convolutional filters through feature interest visualization and gradient analysis, CAPTOR can effectively cluster and adaptively prune the filters associated with unneeded…

Privacy-preserving federated learning for transportation mode prediction based on personal mobility data

Fuxun Yu, Zirui Xu, Zhuwei Qin +1 more

Transportation · High-Confidence Computing · · cited 9× · DOI ↗ · more like this ·

Personal daily mobility trajectories/traces like Google Location Service integrates many valuable information from individuals and could benefit a lot of application scenarios, such as pandemic control and precaution, product recommendation, customized user profile analysis, traffic management in smart cities, etc. However, utilizing such personal mobility data faces many challenges since users’ private information, such as home/work addresses, can be unintentionally leaked. In this work, we build an FL system for transportation mode prediction based on personal mobility data. Utilizing FL-based training scheme, all user’s data are kept in local without uploading to central nodes, providing high privacy preserving capability. At the same time, we could train accurate DNN models that is close to the centralized training performance. The resulted transportation mode prediction system…

Wideband Spectrum Sensing based on Collaborative Multi-Task Learning

Weishan Zhang, Yue Wang, Fuxun Yu +3 more

Computer Networks and Communications · 2022 IEEE International Conference on Communications Workshops (ICC Workshops) · · cited 8× · DOI ↗ · more like this ·

To deal with the complex wireless conditions in cognitive radios, data-driven learning technologies have been advocated for spectrum sensing. While the most existing learning-based methods are designed for basic single-band and narrow-band circumstances, they may not work well in practical wide-band regimes. Due to the limited sensing capability and hardware constraints of practical secondary users (SUs) devices, individual SUs can only observe a portion of the entire wideband spectrum pool. It is also known as the issue of partial observations, which leads to a heterogeneous multi-task learning problem. To overcome these challenges, this work proposes a novel framework of wideband spectrum sensing via collaborative learning among distributed SUs. Capitalizing on the hierarchical nature of feature extraction in deep neural networks (DNN), we design a novel multi-task DNN architecture to…

Towards Robust Training of Neural Networks by Regularizing Adversarial Gradients

Fuxun Yu, Zirui Xu, Yanzhi Wang +2 more

cs.LG · · cited 8× · arXiv ↗ · more like this ·

In recent years, neural networks have demonstrated outstanding effectiveness in a large amount of applications.However, recent works have shown that neural networks are susceptible to adversarial examples, indicating possible flaws intrinsic to the network structures. To address this problem and improve the robustness of neural networks, we investigate the fundamental mechanisms behind adversarial examples and propose a novel robust training method via regulating adversarial gradients. The regulation effectively squeezes the adversarial gradients of neural networks and significantly increases the difficulty of adversarial example generation.Without any adversarial example involved, the robust training method could generate naturally robust networks, which are near-immune to various types of adversarial examples. Experiments show the naturally robust networks can achieve optimal accuracy…

FalCon: Fine-grained Feature Map Sparsity Computing with Decomposed Convolutions for Inference Optimization

Zirui Xu, Fuxun Yu, Chenxi Liu +3 more

Computer Vision and Pattern Recognition · 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · · cited 7× · DOI ↗ · more like this ·

Many works focus on the model’s static parameter optimization (e.g., filters and weights) for CNN inference acceleration. Compared to parameter sparsity, feature map sparsity is per-input related which has better adaptability. The practical sparsity patterns are non-structural and randomly located on feature maps with non-identical shapes. However, the existing feature map sparsity works take computing efficiency as the primary goal, thereby they can only remove structural sparsity and fail to match the above characteristics. In this paper, we develop a novel sparsity computing scheme called FalCon, which can well adapt to the practical sparsity patterns while still maintaining efficient computing. Specifically, we first propose a decomposed convolution design that enables a fine-grained computing unit for sparsity. Additionally, a decomposed convolution computing optimization…

MASKER

Fuxun Yu, Zirui Xu, Chenchen Liu +1 more

Artificial Intelligence · · cited 7× · DOI ↗ · more like this ·

Benefited from recent artificial intelligence evolution, Automatic Speech Recognition (ASR) technology has achieved enormous performance improvement and wider application. Unfortunately, ASR is also heavily leveraged by speech eavesdropping, where ASR is used to translate large volume of intercepted vocal speech into text content, causing considerable information leakage. In this work, we propose MASKER -- a mobile security enhancement solution to protect the mobile speech data from ASR in eavesdropping. By identifying ASR models' ubiquitous vulnerability, MASKER is designed to generate human imperceptible adversarial noises into the real-time speech on the mobile device (e.g. phone call and voice message). Even the speech data is exposed to eavesdropping during data transmission, the adversarial noises can effectively perturb the ASR process with significant Word Error Rate (WER).…

HAMPER

Zirui Xu, Fuxun Yu, Chenchen Liu +1 more

Artificial Intelligence · · cited 7× · DOI ↗ · more like this ·

Recently, the machine learning technologies have been widely used in cognitive applications such as Automatic Speech Recognition (ASR) and Image Recognition (IR). Unfortunately, these techniques have been massively used in unauthorized audio/image data analysis, causing serious privacy leakage. To address this issue, we propose HAMPER in this work, which is a data encryption framework that protects the audio/image data from unauthorized ASR/IR analysis. Leveraging machine learning models' vulnerability to adversarial examples, HAMPER encrypt the audio/image data with adversarial noises to perturb the recognition results of ASR/IR systems. To deploy the proposed framework in extensive platforms (e.g. mobile devices), HAMPER also take into consideration of computation efficiency, perturbation transferability, as well as data attribute configuration. Therefore, rather than focusing on the…