arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Zebin Yang

34 papers · 2011–2026

2011: 12012: 02013: 02014: 02015: 02016: 12017: 02018: 32019: 12020: 62021: 12022: 02023: 42024: 72025: 52026: 5
2011peak 7 in 20242026

Frequent co-authors

Papers

showing 1–25 of 34 RSS

GAMI-Net: An Explainable Neural Network based on Generalized Additive Models with Structured Interactions

Zebin Yang, Aijun Zhang, Agus Sudjianto

stat.ML · · cited 187× · arXiv ↗ · more like this ·

The lack of interpretability is an inevitable problem when using neural network models in real applications. In this paper, an explainable neural network based on generalized additive models with structured interactions (GAMI-Net) is proposed to pursue a good balance between prediction accuracy and model interpretability. GAMI-Net is a disentangled feedforward network with multiple additive subnetworks; each subnetwork consists of multiple hidden layers and is designed for capturing one main effect or one pairwise interaction. Three interpretability aspects are further considered, including a) sparsity, to select the most significant effects for parsimonious representations; b) heredity, a pairwise interaction could only be included when at least one of its parent main effects exists; and c) marginal clarity, to make main effects and pairwise interactions mutually distinguishable. An…

Enhancing Explainability of Neural Networks through Architecture Constraints

Zebin Yang, Aijun Zhang, Agus Sudjianto

stat.ML · · cited 106× · arXiv ↗ · more like this ·

Prediction accuracy and model explainability are the two most important objectives when developing machine learning algorithms to solve real-world problems. The neural networks are known to possess good prediction performance, but lack of sufficient model interpretability. In this paper, we propose to enhance the explainability of neural networks through the following architecture constraints: a) sparse additive subnetworks; b) projection pursuit with orthogonality constraint; and c) smooth function approximation. It leads to an explainable neural network (xNN) with the superior balance between prediction performance and model interpretability. We derive the necessary and sufficient identifiability conditions for the proposed xNN model. The multiple parameters are simultaneously estimated by a modified mini-batch gradient descent method based on the backpropagation algorithm for…

Unwrapping The Black Box of Deep ReLU Networks: Interpretability, Diagnostics, and Simplification

Agus Sudjianto, William Knauth, Rahul Singh +2 more

cs.LG · · cited 55× · arXiv ↗ · more like this ·

The deep neural networks (DNNs) have achieved great success in learning complex patterns with strong predictive power, but they are often thought of as "black box" models without a sufficient level of transparency and interpretability. It is important to demystify the DNNs with rigorous mathematics and practical tools, especially when they are used for mission-critical applications. This paper aims to unwrap the black box of deep ReLU networks through local linear representation, which utilizes the activation pattern and disentangles the complex network into an equivalent set of local linear models (LLMs). We develop a convenient LLM-based toolkit for interpretability, diagnostics, and simplification of a pre-trained deep ReLU network. We propose the local linear profile plot and other visualization methods for interpretation and diagnostics, and an effective merging strategy for…

LightMamba: Efficient Mamba Acceleration on FPGA with Quantization and Hardware Co-design

Renjie Wei, Songqiang Xu, Linfeng Zhong +5 more

cs.CL · · cited 29× · arXiv ↗ · more like this ·

State space models (SSMs) like Mamba have recently attracted much attention. Compared to Transformer-based large language models (LLMs), Mamba achieves linear computation complexity with the sequence length and demonstrates superior performance. However, Mamba is hard to accelerate due to the scattered activation outliers and the complex computation dependency, rendering existing LLM accelerators inefficient. In this paper, we propose LightMamba that co-designs the quantization algorithm and FPGA accelerator architecture for efficient Mamba inference. We first propose an FPGA-friendly post-training quantization algorithm that features rotation-assisted quantization and power-of-two SSM quantization to reduce the majority of computation to 4-bit. We further design an FPGA accelerator that partially unrolls the Mamba computation to balance the efficiency and hardware costs. Through…

Prediction-Based Multi-Objective Optimization for Oil Purchasing and Distribution with the NSGA-II Algorithm

Lean Yu, Zebin Yang, Ling Tang

Artificial Intelligence · International Journal of Information Technology & Decision Making · · cited 22× · DOI ↗ · more like this ·

Due to the uncertainty in oil markets, this paper proposes a novel approach for oil purchasing and distribution optimization by incorporating price and demand prediction, i.e., the prediction-based oil purchasing-and-distribution optimization model. In particular, the proposed method bridges the latest information technology and decision-making technique by introducing the recently proposed information technology (i.e., extreme learning machine (ELM)) into the oil purchasing-and-distribution optimization model. Two main steps are involved: market prediction and planning optimization in the proposed model. In market prediction, the ELM technique is employed to provide fast training time and accurate forecasting results for oil prices and demands. In planning optimization, two objectives of general profit maximization and inventory risk minimization are considered; and the most popular…

Hyperparameter Optimization via Sequential Uniform Designs

Zebin Yang, Aijun Zhang

cs.LG · · cited 21× · arXiv ↗ · more like this ·

Hyperparameter optimization (HPO) plays a central role in the automated machine learning (AutoML). It is a challenging task as the response surfaces of hyperparameters are generally unknown, hence essentially a global optimization problem. This paper reformulates HPO as a computer experiment and proposes a novel sequential uniform design (SeqUD) strategy with three-fold advantages: a) the hyperparameter space is adaptively explored with evenly spread design points, without the need of expensive meta-modeling and acquisition optimization; b) the batch-by-batch design points are sequentially generated with parallel processing support; c) a new augmented uniform design algorithm is developed for the efficient real-time generation of follow-up design points. Extensive experiments are conducted on both global optimization tasks and HPO applications. The numerical results show that the…

Quantile estimators with orthogonal pinball loss function

Lean Yu, Zebin Yang, Ling Tang

Statistics and Probability · Journal of Forecasting · · cited 12× · DOI ↗ · more like this ·

Abstract To guarantee stable quantile estimations even for noisy data, a novel loss function and novel quantile estimators are developed, by introducing the effective concept of orthogonal loss considering the noise in both response and explanatory variables. In particular, the pinball loss used in classical quantile estimators is improved into novel orthogonal pinball loss (OPL) by replacing vertical loss by orthogonal loss. Accordingly, linear quantile regression (QR) and support vector machine quantile regression (SVMQR) can be respectively extended into novel OPL‐based QR and OPL‐based SVMQR models. The empirical study on 10 publicly available datasets statistically verifies the superiority of the two OPL‐based models over their respective original forms in terms of prediction accuracy and quantile property, especially for extreme quantiles. Furthermore, the novel OPL‐based SVMQR…

Hyperparameter tuning methods in automated machine learning

Zebin Yang, Aijun Zhang

Artificial Intelligence · Scientia Sinica Mathematica · · cited 8× · DOI ↗ · more like this ·

In this paper, we make a comprehensive review of the challenging task of hyperparameter optimization in automated machine learning. The commonly used hyperparameter optimization methods include single-shot sampling strategies, e.g., grid search, random search and sequential strategies where new trials are gradually augmented based on existing information, including Bayesian optimization, evolutionary algorithms, reinforcement learning-based methods.We find the sequential number-theoretic optimization (SNTO) algorithm proposed by Kai-Tai Fang and Yuan Wang in 1990 can also be applied in hyperparameter optimization, where sequential uniform designs are utilized to search the global optima in complex response surfaces. For illustration, various hyperparameter optimization methods are tested with two widely-used machine learning models including the support vector machine (SVM) and extreme…

Efficient Model-Free Subsampling Method for Massive Data

Zheng Zhou, Zebin Yang, Aijun Zhang +1 more

Artificial Intelligence · Technometrics · · cited 7× · DOI ↗ · more like this ·

Subsampling plays a crucial role in tackling problems associated with the storage and statistical learning of massive datasets. However, most existing subsampling methods are model-based, which means their performances can drop significantly when the underlying model is misspecified. Such an issue calls for model-free subsampling methods that are robust under diverse model specifications. Recently, several model-free subsampling methods have been developed. However, the computing time of these methods grows explosively with the sample size, making them impractical for handling massive data. In this article, an efficient model-free subsampling method is proposed, which segments the original data into some regular data blocks and obtains subsamples from each data block by the data-driven subsampling method. Compared with existing model-free subsampling methods, the proposed method has a…

PiML Toolbox for Interpretable Machine Learning Model Development and Diagnostics

Agus Sudjianto, Aijun Zhang, Zebin Yang +2 more

cs.LG · · cited 6× · arXiv ↗ · more like this ·

PiML (read $π$-ML, /`pai`em`el/) is an integrated and open-access Python toolbox for interpretable machine learning model development and model diagnostics. It is designed with machine learning workflows in both low-code and high-code modes, including data pipeline, model training and tuning, model interpretation and explanation, and model diagnostics and comparison. The toolbox supports a growing list of interpretable models (e.g. GAM, GAMI-Net, XGB1/XGB2) with inherent local and/or global interpretability. It also supports model-agnostic explainability tools (e.g. PFI, PDP, LIME, SHAP) and a powerful suite of model-agnostic diagnostics (e.g. weakness, reliability, robustness, resilience, fairness). Integration of PiML models and tests to existing MLOps platforms for quality assurance are enabled by flexible high-code APIs. Furthermore, PiML toolbox comes with a comprehensive user…

ProPD: Dynamic Token Tree Pruning and Generation for LLM Parallel Decoding

Shuzhang Zhong, Zebin Yang, Meng Li +3 more

cs.LG · · cited 2× · arXiv ↗ · more like this ·

Recent advancements in generative large language models (LLMs) have significantly boosted the performance in natural language processing tasks. However, their efficiency is hampered by the inherent limitations in autoregressive token generation. While parallel decoding with token tree verification, e.g., Medusa, has been proposed to improve decoding parallelism and efficiency, it often struggles with maintaining contextual relationships due to its independent token prediction approach and incurs significant verification overhead, especially with large tree sizes and batch processing. In this paper, we propose ProPD, an efficient LLM parallel decoding framework based on dynamic token tree pruning and generation. ProPD features an advanced early pruning mechanism to efficiently eliminate unpromising token sequences to improve verification efficiency. Additionally, it introduces a dynamic…

Single-Index Model Tree

Agus Sudjianto, Zebin Yang, Aijun Zhang

Artificial Intelligence · IEEE Transactions on Knowledge and Data Engineering · · cited 2× · DOI ↗ · more like this ·

In this paper, a novel single-index model tree (SIMTree) is proposed. It adopts the recursive partitioning strategy and each data segment is modeled by a single-index model (SIM), which is a flexible extension of linear regression with non-parametric link functions. The proposed SIMTree has 2 major advantages: a) with only a few leaf nodes, it can achieve competitive predictive performance compared to complicated black-box models; b) SIMs fitted on each local data segment are intrinsically interpretable. However, using conventional techniques to build such a SIMTree can be extremely time-consuming. SIM estimation typically involves iterative optimization via Newton-type algorithms; such a resource-intensive estimation procedure is repeatedly used for fitting leaf node SIMs and the search of optimal splits. To make the computation burden affordable, an effective training algorithm is…

Explainable Recommendation Systems by Generalized Additive Models with Manifest and Latent Interactions

Yifeng Guo, Yu Su, Zebin Yang +1 more

cs.LG · · cited 2× · arXiv ↗ · more like this ·

In recent years, the field of recommendation systems has attracted increasing attention to developing predictive models that provide explanations of why an item is recommended to a user. The explanations can be either obtained by post-hoc diagnostics after fitting a relatively complex model or embedded into an intrinsically interpretable model. In this paper, we propose the explainable recommendation systems based on a generalized additive model with manifest and latent interactions (GAMMLI). This model architecture is intrinsically interpretable, as it additively consists of the user and item main effects, the manifest user-item interactions based on observed features, and the latent interaction effects from residuals. Unlike conventional collaborative filtering methods, the group effect of users and items are considered in GAMMLI. It is beneficial for enhancing the model…

Unveiling anisotropic creep deformation of 2195 Al–Li alloy during creep aging process: a crystal plasticity finite element analysis

Chao Lei, Zebin Yang, Jun Liu +3 more

Education · Journal of Materials Research and Technology · · cited 1× · DOI ↗ · more like this ·

A thorough comprehension of the origin and mechanism underlying anisotropic creep deformation is crucial for improving the predictive accuracy and control efficiency of springback in high-strength lightweight aluminum alloy panel components during creep age forming. By leveraging data from uniaxial tensile creep aging tests and electron backscatter diffraction (EBSD) characterization, a crystal plasticity finite element model based on dislocation density has been developed to simulate the creep deformation behavior of 2195 Al–Li alloy during the creep aging process. The effects of texture strength and dislocation density in the plate blank on anisotropic creep deformation are investigated. The findings indicate that a strong initial texture is the primary factor contributing to anisotropy in creep age forming, and the predominant deformation mechanism is the dislocation mechanism, which…

Inherently Interpretable Tree Ensemble Learning

Zebin Yang, Agus Sudjianto, Xiaoming Li +1 more

stat.ML · · cited 1× · arXiv ↗ · more like this ·

Tree ensemble models like random forests and gradient boosting machines are widely used in machine learning due to their excellent predictive performance. However, a high-performance ensemble consisting of a large number of decision trees lacks sufficient transparency and explainability. In this paper, we demonstrate that when shallow decision trees are used as base learners, the ensemble learning algorithms can not only become inherently interpretable subject to an equivalent representation as the generalized additive models but also sometimes lead to better generalization performance. First, an interpretation algorithm is developed that converts the tree ensemble into the functional ANOVA representation with inherent interpretability. Second, two strategies are proposed to further enhance the model interpretability, i.e., by adding constraints in the model training stage and post-hoc…

User Mining Algorithm Based On PageRank Modeling Improvement and Comprehensive Influence Evaluation

Fuyun He, Binbin Huang, Yanting Shen +1 more

Information Systems · 2018 IEEE 4th Information Technology and Mechatronics Engineering Conference (ITOEC) · · cited 1× · DOI ↗ · more like this ·

Microblog has become the most popular medium of mass information dissemination. Users can know more friends in this virtual network through microblog's recommendation function. For example, Sina microblog, users can find like-minded people through the "add friends - interest" module, at present, microblog only recommends authenticated users or users' friends, and does not recommend most influential users to them. In view of the shortcomings mentioned above, this paper improves the PageRank modeling and adds the assessment of interaction and personal influence, thus improving the accuracy and comprehensiveness of the influential users' mining. In order to verify the rationality and effectiveness of the algorithm, the real user data on Sina microblog based on Python grabbing were used for algorithm simulation, the experimental results show this algorithm has obvious advantages in terms of…

MAServeBench: Benchmarking Multi-Agent LLM Serving Systems with Hierarchical Collaboration Workloads

Shengxuan Qiu, Haochen Huang, Shuzhang Zhong +3 more

cs.MA · · cited 0× · arXiv ↗ · more like this ·

Large language model (LLM)-based multi-agent systems (MAS) coordinate specialized agents to solve complex tasks, turning each user request into interdependent model and tool operations. Unlike independent requests, these operations become ready at different times and consume evolving context as agents coordinate and exchange results. This creates changing patterns of concurrency, computation, and memory demand that can expose different bottlenecks in LLM serving hardware. Yet benchmarking these effects requires capturing the collaboration rules that generate serving workloads, beyond the requests observed during execution. We present MAServeBench, a benchmark that treats multi-agent collaboration as a configurable dimension of serving workloads for LLM serving and hardware evaluation. Its composable stages capture how agents coordinate and exchange context in documented MAS workflows…

Knowledge as Power: The Role of Organizational Learning in Driving Effective Public Policy Implementation

Zebin YANG, Zhenyang ZHANG

Political Science and International Relations · China Public Administration Review · · cited 0× · DOI ↗ · more like this ·

Abstract It is widely acknowledged that organizational learning is essential in the policy-making process, as it helps to refine policy design, align stakeholder interests, and enhance the quality of policy decisions in response to dynamic governance environments. However, does organizational learning also play a critical role in policy implementation, and if so, how does it contribute to effective policy implementation? To address these key questions, this study constructs an integrated “organizational learning & knowledge creation” theoretical framework, drawing on organizational learning theory and knowledge production theory with a focus on the dynamic transformation between tacit and explicit knowledge and the multi-stage knowledge production process, to analyze the processes, mechanisms and practical impacts of organizational learning conducted by local governments during policy…

Stable Subsampling under Model Misspecification and Covariate Shift

Jinjing Yang, Shaohua Xu, Zebin Yang +2 more

Statistics and Probability · ACM Transactions on Knowledge Discovery from Data · · cited 0× · DOI ↗ · more like this ·

The presence of covariate shift between training and test datasets, coupled with model misspecification, can lead to instability in regression predictions across diverse datasets. Meanwhile, training complex models with massive data imposes significant computational burden. In this article, we present a novel model-free subsampling algorithm for stable prediction, which employs uniform design and confounder balancing methods. Our subsampling algorithm aims to find the nearest neighbor subsampling points of uniform design with the goal of minimizing global stability loss, thereby reducing the data volume while achieving stable predictions. Theoretic analyses show that the uniform measure minimizes the maximum integrated mean square error (MIMSE) and the global stability loss evaluates the independence among variables in each candidate MIMSE-optimal subsampled sets. Simulation studies…

Improved Crayfish Optimization Algorithm for Mobile Robot Path Planning Optimization

Chao Huang, Zebin Yang

Computer Vision and Pattern Recognition · · cited 0× · DOI ↗ · more like this ·

Regarding the issues of non-smooth paths, unstable obstacle avoidance, and getting stuck in local optima in mobile robot path planning, a new Adaptive Crayfish Optimization Algorithm (ACOA) is proposed. The algorithm improves robot path planning. First, a Piecewise Chaotic Map is used to set up the population. It adds diversity and randomness to the population and helps the algorithm search better globally. Finally, a special fitness function is designed. It allows robots to avoid obstacles better and creates smoother paths. Tests show that the improved algorithm works well in different complex environments. It gives shorter paths, faster convergence, and fewer turns. Compared with the COA algorithm, the ACOA algorithm minimizes the path length by 3.9% and 5.3% in the two maps, and the inflection points are reduced by 40% and 33%, respectively. It also smooths It also smooths paths and…

A Joint Generative Discriminant Network for Person Re-identification with Limited Computing Resources

Junping Yao, Hao Wang, Xiaojun Li +2 more

Computer Vision and Pattern Recognition · · cited 0× · DOI ↗ · more like this ·

Traditional methods for Person Re-identification (Re-ID) often achieve high accuracy at the cost of huge computational resources, and their application is severely limited under low computing resources. Previous studies have shown that reducing the number of model parameters often significantly degrades the performance of the model. Based on this, we constructs a lightweight joint generative discriminant network, that the maximum pooling layer after group convolution point by point, as well as the global pooling layer and fully connected layer after channel shuffle in the fourth stage, are removed. Then, a convolutional layer is added at the end of the fourth stage, and use an adaptive pooling layer to output a feature map of the expected dimension, while applying knowledge distillation in training to improve the performance of the network. We conducted experiments on the Market-1501…

MCUBERT: Memory-Efficient BERT Inference on Commodity Microcontrollers

Zebin Yang, Renze Chen, Taiqiang Wu +5 more

cs.LG · · cited 0× · arXiv ↗ · more like this ·

In this paper, we propose MCUBERT to enable language models like BERT on tiny microcontroller units (MCUs) through network and scheduling co-optimization. We observe the embedding table contributes to the major storage bottleneck for tiny BERT models. Hence, at the network level, we propose an MCU-aware two-stage neural architecture search algorithm based on clustered low-rank approximation for embedding compression. To reduce the inference memory requirements, we further propose a novel fine-grained MCU-friendly scheduling strategy. Through careful computation tiling and re-ordering as well as kernel design, we drastically increase the input sequence lengths supported on MCUs without any latency or accuracy penalty. MCUBERT reduces the parameter size of BERT-tiny and BERT-mini by 5.7$\times$ and 3.0$\times$ and the execution memory by 3.5$\times$ and 4.3$\times$, respectively. MCUBERT…

FastQuery: Communication-efficient Embedding Table Query for Private LLMs inference

Chenqi Lin, Tianshi Xu, Zebin Yang +3 more

Artificial Intelligence · · cited 0× · DOI ↗ · more like this ·

With the fast evolution of large language models (LLMs), privacy concerns with user queries arise as they may contain sensitive information. Private inference based on homomorphic encryption (HE) has been proposed to protect user query privacy. However, private embedding table query has to be formulated as a HE-based matrix-vector multiplication problem and suffers from enormous computation and communication overhead. We observe the overhead mainly comes from the neglect of 1) the one-hot nature of user queries and 2) the robustness of the embedding table to low bit-width quantization noise. Hence, in this paper, we propose a private embedding table query optimization framework, dubbed FastQuery. FastQuery features a communication-aware embedding table quantization algorithm and a one-hot-aware dense packing algorithm to simultaneously reduce both the computation and communication…

FastQuery: Communication-efficient Embedding Table Query for Private LLM Inference

Chenqi Lin, Tianshi Xu, Zebin Yang +3 more

cs.CR · · cited 0× · arXiv ↗ · more like this ·

With the fast evolution of large language models (LLMs), privacy concerns with user queries arise as they may contain sensitive information. Private inference based on homomorphic encryption (HE) has been proposed to protect user query privacy. However, a private embedding table query has to be formulated as a HE-based matrix-vector multiplication problem and suffers from enormous computation and communication overhead. We observe the overhead mainly comes from the neglect of 1) the one-hot nature of user queries and 2) the robustness of the embedding table to low bit-width quantization noise. Hence, in this paper, we propose a private embedding table query optimization framework, dubbed FastQuery. FastQuery features a communication-aware embedding table quantization algorithm and a one-hot-aware dense packing algorithm to simultaneously reduce both the computation and communication…

A mask-guided generation learning loss optimization method for person re-identification

Zebin Yang, Junping Yao, Xiaojun Li +1 more

Computer Vision and Pattern Recognition · · cited 0× · DOI ↗ · more like this ·

Traditional person re-identification (ReID) methods based on generative adversarial networks often neglect the impact of background clutter during the image generation stage of data augmentation, which can have a negative effect on the accuracy of subsequent recognition results. To address this problem, this paper proposes a mask-guided generation learning loss optimization method, which consists of reconstructed image loss optimization and composite image loss optimization. By optimizing the loss function in the image generation stage, the proposed method generates more real images of the person’s body part to eliminate the impact of background clutter on recognition results in subsequent stages. The reconstructed image loss optimization uses a body mask to construct an attention transfer matrix, mapping the original image into a new feature map containing person-related but…