arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

HAOLi

183 papers · 2003–2026

2003: 12004: 12005: 22006: 02007: 22008: 22009: 12010: 32011: 82012: 32013: 32014: 42015: 62016: 12017: 62018: 92019: 62020: 152021: 82022: 72023: 132024: 152025: 282026: 39
2003peak 39 in 20262026

Frequent co-authors

Papers

showing 1–25 of 183 RSS

BinaryBERT: Pushing the Limit of BERT Quantization

Haoli Bai, Wei Zhang, Lu Hou +6 more

cs.CL · · cited 282× · arXiv ↗ · more like this ·

The rapid development of large pre-trained language models has greatly increased the demand for model compression techniques, among which quantization is a popular solution. In this paper, we propose BinaryBERT, which pushes BERT quantization to the limit by weight binarization. We find that a binary BERT is hard to be trained directly than a ternary counterpart due to its complex and irregular loss landscape. Therefore, we propose ternary weight splitting, which initializes BinaryBERT by equivalently splitting from a half-sized ternary network. The binary model thus inherits the good performance of the ternary one, and can be further enhanced by fine-tuning the new architecture after splitting. Empirical results show that our BinaryBERT has only a slight performance drop compared with the full-precision model while being 24x smaller, achieving the state-of-the-art compression results…

A High Capacity Multi-Level Approach for Reversible Data Hiding in Encrypted Images

Haoli Ge, Yan Chen, Zhenxing Qian +1 more

Computer Vision and Pattern Recognition · IEEE Transactions on Circuits and Systems for Video Technology · · cited 103× · DOI ↗ · more like this ·

This paper proposes a novel method of reversible data hiding in encrypted images (RDH-EI). We first provide an RDH-EI approach using single-level embedding, in which three parties are involved, including an image owner, a data hider, and a recipient. The image owner encrypts an original image into a ciphertext image. After dividing the original image into blocks, the owner pseudo-randomly permutes all blocks by a permutation key. With an encryption key, the image owner further encrypts the contents of all blocks using a stream cipher algorithm, during which pixels inside each block share the same stream bytes. Once the encrypted image is uploaded onto the server, a data hider embeds additional messages into the ciphertext. The data hider divides the encrypted image into blocks and selects peak pixels from each block using an embedding key. With the peak pixels, the data hider embeds an…

Few Shot Network Compression via Cross Distillation

Haoli Bai, Jiaxiang Wu, Irwin King +1 more

cs.LG · · cited 71× · arXiv ↗ · more like this ·

Model compression has been widely adopted to obtain light-weighted deep neural networks. Most prevalent methods, however, require fine-tuning with sufficient training data to ensure accuracy, which could be challenged by privacy and security issues. As a compromise between privacy and performance, in this paper we investigate few shot network compression: given few samples per class, how can we effectively compress the network with negligible performance drop? The core challenge of few shot network compression lies in high estimation errors from the original network during inference, since the compressed network can easily over-fits on the few training instances. The estimation errors could propagate and accumulate layer-wisely and finally deteriorate the network output. To address the problem, we propose cross distillation, a novel layer-wise knowledge distillation approach. By…

Towards Efficient Post-training Quantization of Pre-trained Language Models

Haoli Bai, Lu Hou, Lifeng Shang +3 more

cs.CL · · cited 62× · arXiv ↗ · more like this ·

Network quantization has gained increasing attention with the rapid growth of large pre-trained language models~(PLMs). However, most existing quantization methods for PLMs follow quantization-aware training~(QAT) that requires end-to-end training with full access to the entire dataset. Therefore, they suffer from slow training, large memory overhead, and data security issues. In this paper, we study post-training quantization~(PTQ) of PLMs, and propose module-wise quantization error minimization~(MREM), an efficient solution to mitigate these issues. By partitioning the PLM into multiple modules, we minimize the reconstruction error incurred by quantization for each module. In addition, we design a new model parallel training strategy such that each module can be trained locally on separate computing devices without waiting for preceding modules, which brings nearly the theoretical…

MoPE-CLIP: Structured Pruning for Efficient Vision-Language Models with Module-wise Pruning Error Metric

Haokun Lin, Haoli Bai, Zhili Liu +5 more

cs.CV · · cited 49× · arXiv ↗ · more like this ·

Vision-language pre-trained models have achieved impressive performance on various downstream tasks. However, their large model sizes hinder their utilization on platforms with limited computational resources. We find that directly using smaller pre-trained models and applying magnitude-based pruning on CLIP models leads to inflexibility and inferior performance. Recent efforts for VLP compression either adopt uni-modal compression metrics resulting in limited performance or involve costly mask-search processes with learnable masks. In this paper, we first propose the Module-wise Pruning Error (MoPE) metric, accurately assessing CLIP module importance by performance decline on cross-modal tasks. Using the MoPE metric, we introduce a unified pruning framework applicable to both pre-training and task-specific fine-tuning compression stages. For pre-training, MoPE-CLIP effectively…

Neural Relational Topic Models for Scientific Article Analysis

Haoli Bai, Zhuangbin Chen, Michael R. Lyu +2 more

Artificial Intelligence · · cited 49× · DOI ↗ · more like this ·

Topic modelling and citation recommendation of scientific articles are important yet challenging research problems in scientific article analysis. In particular, the inference on coherent topics can be easily affected by irrelevant contents in articles. Meanwhile, the extreme sparsity of citation networks brings difficulty to a valid citation recommendation. Intuitively, articles with similar topics are more likely to cite each other, and cited articles tend to share similar themes. Motivated from this intuition, we aim to boost the performance of both topic modelling and citation recommendation by effectively leverage this underlying correlation between latent topics and citation networks. To this end, we propose a novel Bayesian deep generative model termed as Neural Relational Topic Model (NRTM), which is composed with a Stacked Variational Auto-Encoder (SVAE) and a multilayer…

On the Domination Number of Generalized Petersen Graphs P(ck,k)

Haoli Wang, Xirong Xu, Yuansheng Yang +1 more

math.CO · · cited 47× · arXiv ↗ · more like this ·

Let $G=(V(G),E(G))$ be a simple connected and undirected graph with vertex set $V(G)$ and edge set $E(G)$. A set $S \subseteq V(G)$ is a $dominating$ $set$ if for each $v \in V(G)$ either $v \in S$ or $v$ is adjacent to some $w \in S$. That is, $S$ is a dominating set if and only if $N[S]=V(G)$. The domination number $γ(G)$ is the minimum cardinalities of minimal dominating sets. In this paper, we give an improved upper bound on the domination number of generalized Petersen graphs $P(ck,k)$ for $c\geq 3$ and $k\geq 3$. We also prove that $γ(P(4k,k))=2k+1$ for even $k$, $γ(P(5k,k))=3k$ for all $k\geq 1$, and $γ(P(6k,k))=\lceil\frac{10k}{3}\rceil$ for $k\geq 1$ and $k\neq 2$.

RTN: Reparameterized Ternary Network

Yuhang Li, Xin Dong, Sai Qian Zhang +3 more

cs.LG · · cited 36× · arXiv ↗ · more like this ·

To deploy deep neural networks on resource-limited devices, quantization has been widely explored. In this work, we study the extremely low-bit networks which have tremendous speed-up, memory saving with quantized activation and weights. We first bring up three omitted issues in extremely low-bit networks: the squashing range of quantized values; the gradient vanishing during backpropagation and the unexploited hardware acceleration of ternary networks. By reparameterizing quantized activation and weights vector with full precision scale and offset for fixed ternary vector, we decouple the range and magnitude from the direction to extenuate the three issues. Learnable scale and offset can automatically adjust the range of quantized values and sparsity without gradient vanishing. A novel encoding and computation pat-tern are designed to support efficient computing for our reparameterized…

Structured Inference for Recurrent Hidden Semi-markov Model

Hao Liu, Lirong He, Haoli Bai +3 more

Signal Processing · · cited 34× · DOI ↗ · more like this ·

Segmentation and labeling for high dimensional time series is an important yet challenging task in a number of applications, such as behavior understanding and medical diagnosis. Recent advances to model the nonlinear dynamics in such time series data, has suggested to involve recurrent neural networks into Hidden Markov Models. However, this involvement has caused the inference procedure much more complicated, often leading to intractable inference, especially for the discrete variables of segmentation and labeling. To achieve both flexibility and tractability in modeling nonlinear dynamics of discrete variables, we present a structured and stochastic sequential neural network (SSNN), which composes with a generative network and an inference network. In detail, the generative network aims to not only capture the long-term dependencies but also model the uncertainty of the segmentation…

Comparison of nature of science representations in five Chinese high school physics textbooks

Haoli Zhuang, Yang Xiao, Qiaoyi Liu +3 more

Education · International Journal of Science Education · · cited 33× · DOI ↗ · more like this ·

Formal understanding of the nature of science (NOS) has been considered to be a major contributor to nurturing students’ scientific literacy. In China, this view has been endorsed in the new standard for high school physics curriculum, which has guided the development of the new generation physics textbooks. Following the analytical framework established in the recent literature, this study evaluates the NOS representations in five textbooks approved by the Chinese Ministry of Education regarding the extent, the manner (explicit vs. implicit), the accuracy and completeness (from naïve to informed), and the overall consistency of different aspects of NOS. The results show that the representations of NOS in all five physics textbooks are far from satisfactory. The aspects of NOS are fully represented by only three textbooks. For all textbooks, the NOS representations are distributed…

Deep Interaction Machine

Feng Yu, Zhaocheng Liu, Qiang Liu +3 more

Information Systems · · cited 32× · DOI ↗ · more like this ·

Click-Through Rate (CTR) prediction is a crucial task for various online applications, such as recommendation and online advertising. The task of CTR prediction is to predict the probability of users' clicking behaviors, with high-dimensional input features. To avoid heavy handcrafted feature engineering, the core topic of CTR prediction is the automatic interactions of the input features. Factorization Machine (FM) is an effective approach for modeling second-order feature interactions. Recently, FM has been extended for modeling higher-order feature interactions, such as xDeepFM and Higher-Order Factorization Machine (HOFM). However, these approaches are with either high complexity or iterative computation consuming much time and space. To overcome above problems, we express arbitrary-order FM in the form of power sums according to Newton's identities. Accordingly, we propose a novel…

Double Sparse Deep Reinforcement Learning via Multilayer Sparse Coding and Nonconvex Regularized Pruning

Haoli Zhao, Jiqiang Wu, Zhenni Li +2 more

Artificial Intelligence · IEEE Transactions on Cybernetics · · cited 29× · DOI ↗ · more like this ·

Deep reinforcement learning (DRL), which highly depends on the data representation, has shown its potential in many practical decision-making problems. However, the process of acquiring representations in DRL is easily affected by interference from models, and moreover leaves unnecessary parameters, leading to control performance reduction. In this article, we propose a double sparse DRL via multilayer sparse coding and nonconvex regularized pruning. To alleviate interference in DRL, we propose a multilayer sparse-coding-structural network to obtain deep sparse representation for control in reinforcement learning. Furthermore, we employ a nonconvex log regularizer to promote strong sparsity, efficiently removing the unnecessary weights with a regularizer-based pruning scheme. Hence, a double sparse DRL algorithm is developed, which can not only learn deep sparse representation to reduce…

Dynamically pruning segformer for efficient semantic segmentation

Haoli Bai, Hongda Mao, Dinesh Nair

cs.CV · · cited 29× · arXiv ↗ · more like this ·

As one of the successful Transformer-based models in computer vision tasks, SegFormer demonstrates superior performance in semantic segmentation. Nevertheless, the high computational cost greatly challenges the deployment of SegFormer on edge devices. In this paper, we seek to design a lightweight SegFormer for efficient semantic segmentation. Based on the observation that neurons in SegFormer layers exhibit large variances across different images, we propose a dynamic gated linear layer, which prunes the most uninformative set of neurons based on the input instance. To improve the dynamically pruned SegFormer, we also introduce two-stage knowledge distillation to transfer the knowledge within the original teacher to the pruned student network. Experimental results show that our method can significantly reduce the computation overhead of SegFormer without an apparent performance drop.…

Ledgerdata Refiner: A Powerful Ledger Data Query Platform for Hyperledger Fabric

Ence Zhou, Haoli Sun, Bingfeng Pi +3 more

cs.DC · · cited 29× · arXiv ↗ · more like this ·

Blockchain is one of the most popular distributed ledger technologies. It can solve the trust issue among enterprises. Hyperledger Fabric is a permissioned blockchain aiming at enterprise-grade business applications. However, compared to traditional distributed database solutions, one issue of blockchain based application development is the limited data access. For Fabric, the ledger data can only be retrieved by limited interfaces provided by Fabric SDKs or chaincode. In order to meet the requirements of data query and provide flexible query functions for real applications built on Fabric, this paper proposed a ledger data query platform called Ledgerdata Refiner. With ledger data analysis middleware, we provide sufficient interfaces for users to retrieve block or transaction efficiently. It is also able to track historical operations for any specific state. In addition, schemas of…

Trusted artificial intelligence for environmental assessments: An explainable high-precision model with multi-source big data

Haoli Xu, Xing Yang, Yihua Hu +9 more

Artificial Intelligence · Environmental Science and Ecotechnology · · cited 23× · DOI ↗ · more like this ·

Environmental assessments are critical for ensuring the sustainable development of human civilization. The integration of artificial intelligence (AI) in these assessments has shown great promise, yet the "black box" nature of AI models often undermines trust due to the lack of transparency in their decision-making processes, even when these models demonstrate high accuracy. To address this challenge, we evaluated the performance of a transformer model against other AI approaches, utilizing extensive multivariate and spatiotemporal environmental datasets encompassing both natural and anthropogenic indicators. We further explored the application of saliency maps as a novel explainability tool in multi-source AI-driven environmental assessments, enabling the identification of individual indicators' contributions to the model's predictions. We find that the transformer model outperforms…

M-NAS: Meta Neural Architecture Search

Jiaxing Wang, Jiaxiang Wu, Haoli Bai +1 more

Artificial Intelligence · Proceedings of the AAAI Conference on Artificial Intelligence · · cited 23× · DOI ↗ · more like this ·

Neural Architecture Search (NAS) has recently outperformed hand-crafted networks in various areas. However, most prevalent NAS methods only focus on a pre-defined task. For a previously unseen task, the architecture is either searched from scratch, which is inefficient, or transferred from the one obtained on some other task, which might be sub-optimal. In this paper, we investigate a previously unexplored problem: whether a universal NAS method exists, such that task-aware architectures can be effectively generated? Towards this problem, we propose Meta Neural Architecture Search (M-NAS). To obtain task-specific architectures, M-NAS adopts a task-aware architecture controller for child model generation. Since optimal weights for different tasks and architectures span diversely, we resort to meta-learning, and learn meta-weights that efficiently adapt to a new task on the corresponding…

Efficient Bitwidth Search for Practical Mixed Precision Neural Network

Yuhang Li, Wei Wang, Haoli Bai +3 more

cs.LG · · cited 23× · arXiv ↗ · more like this ·

Network quantization has rapidly become one of the most widely used methods to compress and accelerate deep neural networks. Recent efforts propose to quantize weights and activations from different layers with different precision to improve the overall performance. However, it is challenging to find the optimal bitwidth (i.e., precision) for weights and activations of each layer efficiently. Meanwhile, it is yet unclear how to perform convolution for weights and activations of different precision efficiently on generic hardware platforms. To resolve these two issues, in this paper, we first propose an Efficient Bitwidth Search (EBS) algorithm, which reuses the meta weights for different quantization bitwidth and thus the strength for each candidate precision can be optimized directly w.r.t the objective without superfluous copies, reducing both the memory and computational cost…

A New Approach on Task Offloading Scheduling for Application of Mobile Edge Computing

Yu‐ya Cui, Degan Zhang, Ting Zhang +2 more

Computer Networks and Communications · · cited 21× · DOI ↗ · more like this ·

In mobile edge computing(MEC), application partitioning can split the executions into local and edge server parts. Optimal partitioning will allow mobile devices to obtain the highest benefit from Mobile Edge Computing (MEC). In this paper, a new approach on task offloading scheduling for application of mobile edge computing is proposed. We divide the computing task into several subtasks. Then, we analyze the overhead of the system, and propose a fine-grained strategy for task scheduling and offloading in a multi-user MEC system. For reducing the energy consumption and delay, the computation offloading is considered as a constrained multi-objective optimization problem (CMOP), which can be solved by an improved NSGA-II algorithm. The experimental results show that the proposed algorithm can find a large number of optimal solutions to adjust the corresponding offloading decision…

Plug-and-Play: An Efficient Post-training Pruning Method for Large Language Models

Yingtao Zhang, Haoli Bai, Haokun Lin +3 more

Artificial Intelligence · Preprints.org · · cited 17× · DOI ↗ · more like this ·

With the rapid growth of large language models (LLMs), there is increasing demand for memory and computation in LLMs. Recent efforts on post-training pruning of LLMs aim to reduce the model size and computation requirements, yet the performance is still sub-optimal. In this paper, we present a plug-and-play solution for post-training pruning of LLMs. The proposed solution has two innovative components: 1) Relative Importance and Activations (RIA), a new pruning metric that jointly considers the weight and activations efficiently on LLMs; and 2) Channel Permutation, a new approach to maximally preserve important weights under N:M sparsity. The two proposed components can be readily combined to further enhance the N:M semi-structured pruning of LLMs. Our empirical experiments show that RIA alone can already surpass all existing post-training pruning methods on prevalent LLMs, e.g., LLaMA…

Bayesian Automatic Model Compression

Jiaxing Wang, Haoli Bai, Jiaxiang Wu +1 more

Artificial Intelligence · IEEE Journal of Selected Topics in Signal Processing · · cited 17× · DOI ↗ · more like this ·

Model compression has drawn great attention in deep learning community. A core problem in model compression is to determine the layer-wise optimal compression policy, e.g., the layer-wise bit-width in network quantization. Conventional hand-crafted heuristics rely on human experts and are usually sub-optimal, while recent reinforcement learning based approaches can be inefficient during the exploration of the policy space. In this article, we propose Bayesian automatic model compression (BAMC), which leverages non-parametric Bayesian methods to learn the optimal quantization bit-width for each layer of the network. BAMC is trained in a one-shot manner, avoiding the back and forth (re)-training in reinforcement learning based approaches. Experimental results on various datasets validate that our proposed methods can find reasonable quantization policies efficiently with little accuracy…

A problem of Wang on Davenport constant for the multiplicative semigroup of the quotient ring of $\F_2[x]$

Lizhen Zhang, Haoli Wang, Yongke Qu

math.CO · · cited 16× · arXiv ↗ · more like this ·

Let $\F_q[x]$ be the ring of polynomials over the finite field $\F_q$, and let $f$ be a polynomial of $\F_q[x]$. Let $R=\frac{\F_q[x]}{(f)}$ be a quotient ring of $\F_q[x]$ with $0\neq R\neq \F_q[x]$. Let $\mathcal{S}_R$ be the multiplicative semigroup of the ring $R$, and let ${\rm U}(\mathcal{S}_R)$ be the group of units of $\mathcal{S}_R$. The Davenport constant ${\rm D}(\mathcal{S}_R)$ of the multiplicative semigroup $\mathcal{S}_R$ is the least positive integer $\ell$ such that for any $\ell$ polynomials $g_1,g_2,\ldots,g_{\ell}\in \F_q[x]$, there exists a subset $I\subsetneq [1,\ell]$ with $$\prod\limits_{i\in I} g_i \equiv \prod\limits_{i=1}^{\ell} g_i\pmod f.$$ In this manuscript, we proved that for the case of $q=2$, $${\rm D}({\rm U}(\mathcal{S}_R))\leq {\rm D}(\mathcal{S}_R)\leq {\rm D}({\rm U}(\mathcal{S}_R))+δ_f,$$ where \begin{displaymath} δ_f=\left\{\begin{array}{ll} 0 &…

Accelerated Partially Shared Dictionary Learning With Differentiable Scale-Invariant Sparsity for Multi-View Clustering

Haoli Zhao, Zhenni Li, Wuhui Chen +2 more

Computer Vision and Pattern Recognition · IEEE Transactions on Neural Networks and Learning Systems · · cited 15× · DOI ↗ · more like this ·

Multiview dictionary learning (DL) is attracting attention in multiview clustering due to the efficient feature learning ability. However, most existing multiview DL algorithms are facing problems in fully utilizing consistent and complementary information simultaneously in the multiview data and learning the most precise representation for multiview clustering because of gaps between views. This article proposes an efficient multiview DL algorithm for multiview clustering, which uses the partially shared DL model with a flexible ratio of shared sparse coefficients to excavate both consistency and complementarity in the multiview data. In particular, a differentiable scale-invariant function is used as the sparsity regularizer, which considers the absolute sparsity of coefficients as the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML"…

A Spatiotemporal Information-Driven Cross-Attention Model With Sparse Representation for GNSS NLOS Signal Classification

Kungan Zeng, Zhenni Li, Haoli Zhao +5 more

Artificial Intelligence · IEEE Internet of Things Journal · · cited 13× · DOI ↗ · more like this ·

Global navigation satellite systems (GNSSs) provide efficient positioning services for location-aware Internet of Things (IoT) devices. However, GNSS non-line-of-sight (NLOS) signals can result in severe positioning errors in urban canyon areas. Existing deep-learning-based NLOS signal classification methods cannot appropriately model the spatiotemporal information of NLOS interference, resulting in limited accuracy across multiple locations. This study presents a spatiotemporal information-driven model that can capture environmental characteristics and signal temporal information simultaneously to improve NLOS classification accuracy across multiple locations. First, a visualization analysis of the signal distribution across multiple locations demonstrates the impact of environmental characteristics. In addition, the significance of both the spatial environmental features and the…

TranSlider: Transfer Ensemble Learning from Exploitation to Exploration

Kuo Zhong, Ying Wei, Chun Yuan +2 more

Artificial Intelligence · · cited 13× · DOI ↗ · more like this ·

In transfer learning, what and where to transfer has been widely studied. Nevertheless, the learned transfer strategies are at high risk of over-fitting, especially when only a few annotated instances are available in the target domain. In this paper, we introduce the concept of transfer ensemble learning, a new direction to tackle the over-fitting of transfer strategies. Intuitively, models with different transfer strategies offer various perspectives on what and where to transfer. Therefore a core problem is to search these diversely transferred models for ensemble so as to achieve better generalization. Towards this end, we propose the Transferability Slider (TranSlider) for transfer ensemble learning. By decreasing the transferability, we obtain a spectrum of base models ranging from pure exploitation of the source model to unconstrained exploration for the target domain.…

Video enhancement:content classification and model selection

HaoLi Hu

Computer Vision and Pattern Recognition · TU/e Research Portal · · cited 13× · DOI ↗ · more like this ·

The purpose of video enhancement is to improve the subjective picture quality. The field of video enhancement includes a broad category of research topics, such as removing noise in the video, highlighting some specified features and improving the appearance or visibility of the video content. The common difficulty in this field is how to make images or videos more beautiful, or subjectively better. Traditional approaches involve lots of iterations between subjective assessment experiments and redesigns of algorithm improvements, which are very time consuming. Researchers have attempted to design a video quality metric to replace the subjective assessment, but so far it is not successful. As a way to avoid heuristics in the enhancement algorithm design, least mean square methods have received considerable attention. They can optimize filter coefficients automatically by minimizing the…