Author

Run He

18 papers · 1753–2026

1967peak 6 in 20242026

Frequent co-authors

Papers

showing 1–18 of 18 RSS

DS-AL: A Dual-Stream Analytic Learning for Exemplar-Free Class-Incremental Learning

Huiping Zhuang, Run He, Kai Tong +3 more

cs.LG · 26 Mar 2024 · cited 70× · arXiv ↗ · more like this ·

Class-incremental learning (CIL) under an exemplar-free constraint has presented a significant challenge. Existing methods adhering to this constraint are prone to catastrophic forgetting, far more so than replay-based techniques that retain access to past samples. In this paper, to solve the exemplar-free CIL problem, we propose a Dual-Stream Analytic Learning (DS-AL) approach. The DS-AL contains a main stream offering an analytical (i.e., closed-form) linear solution, and a compensation stream improving the inherent under-fitting limitation due to adopting linear mapping. The main stream redefines the CIL problem into a Concatenated Recursive Least Squares (C-RLS) task, allowing an equivalence between the CIL and its joint-learning counterpart. The compensation stream is governed by a Dual-Activation Compensation (DAC) module. This module re-activates the embedding with a different…

GKEAL: Gaussian Kernel Embedded Analytic Learning for Few-Shot Class Incremental Task

Huiping Zhuang, Zhenyu Weng, Run He +2 more

Artificial Intelligence · June 2023 · cited 55× · DOI ↗ · more like this ·

Few-shot class incremental learning (FSCIL) aims to address catastrophic forgetting during class incremental learning in a few-shot learning setting. In this paper, we approach the FSCIL by adopting analytic learning, a technique that converts network training into linear problems. This is inspired by the fact that the recursive implementation (batch-by-batch learning) of analytic learning gives identical weights to that produced by training on the entire dataset at once. The recursive implementation and the weight-identical property highly resemble the FSCIL setting (phase-by-phase learning) and its goal of avoiding catastrophic forgetting. By bridging the FSCIL with the analytic learning, we propose a Gaussian kernel embedded analytic learning (GKEAL) for FSCIL. The key components of GKEAL include the kernel analytic module which allows the GKEAL to conduct FSCIL in a recursive…

Integrated confusion-diffusion mechanisms for chaos based image encryption

Qiu Run-he, Yun Cao, Fu Yu-Zhen

Computer Vision and Pattern Recognition · October 2011 · cited 12× · DOI ↗ · more like this ·

A new kind of color image encryption algorithm based on integrated confusion-diffusion mechanisms and chaos is put forward. Firstly, generate three values by iterating two-dimensional logistic maps, which are used as the three offset values of RGB components respectively. Secondly, choose three variables from the four-dimensional hyper-chaos to generate four different combinations. Next, XOR operation is conducted between the chosen combination and the original image with three offset values. Thus, integration confusion with diffusion is achieved. The experimental results demonstrate that this algorithm has advantages of large key space and high security, and moreover, it is sensitive to the secret keys, his electronic document is a “live” template.

AFL: A Single-Round Analytic Approach for Federated Learning with Pre-trained Models

Run He, Kai Tong, Di Fang +5 more

cs.LG · May 2024 · cited 2× · arXiv ↗ · more like this ·

In this paper, we introduce analytic federated learning (AFL), a new training paradigm that brings analytical (i.e., closed-form) solutions to the federated learning (FL) with pre-trained models. Our AFL draws inspiration from analytic learning -- a gradient-free technique that trains neural networks with analytical solutions in one epoch. In the local client training stage, the AFL facilitates a one-epoch training, eliminating the necessity for multi-epoch updates. In the aggregation stage, we derive an absolute aggregation (AA) law. This AA law allows a single-round aggregation, reducing heavy communication overhead and achieving fast convergence by removing the need for multiple aggregation rounds. More importantly, the AFL exhibits a property that \textit{invariance to data partitioning}, meaning that regardless of how the full dataset is distributed among clients, the aggregated…

Complex Motion Planning for Quadruped Robots Using Large Language Models

Xiang Zhang, Run He, Kai Tong +4 more

Artificial Intelligence · 19 May 2024 · cited 1× · DOI ↗ · more like this ·

Large language models (LLMs) have shown dominant performance in various language tasks, including code-writing, machine translation, and semantic comprehension. With prompt engineering, LLMs can also comprehend complex tasks and translate them into executable code. These powers offer great potential for controlling the motion of robots. In this paper, we focus on leveraging the ability of LLMs, prompt engineering, and predefined robot action APIs to facilitate high-level motion planning for quadruped robots. With LLMs, we enable the robot to autonomously plan and execute sophisticated actions based on the comprehension of effective prompts. Through various experiments and evaluations, we demonstrate the effectiveness and adaptability of our approach in handling intricate motion tasks. Our research contributes to the advancement of intelligent robotics and paves the way for more…

APFL: Analytic Personalized Federated Learning via Dual-Stream Least Squares

Kejia Fan, Jianheng Tang, Zhirui Yang +8 more

cs.LG · 14 Aug 2025 · cited 0× · arXiv ↗ · more like this ·

Personalized Federated Learning (PFL) has presented a significant challenge to deliver personalized models to individual clients through collaborative training. Existing PFL methods are often vulnerable to non-IID data, which severely hinders collective generalization and then compromises the subsequent personalization efforts. In this paper, to address this non-IID issue in PFL, we propose an Analytic Personalized Federated Learning (APFL) approach via dual-stream least squares. In our APFL, we use a foundation model as a frozen backbone for feature extraction. Subsequent to the feature extractor, we develop dual-stream analytic models to achieve both collective generalization and individual personalization. Specifically, our APFL incorporates a shared primary stream for global generalization across all clients, and a dedicated refinement stream for local personalization of each…

L3A: Label-Augmented Analytic Adaptation for Multi-Label Class Incremental Learning

Xiang Zhang, Run He, Jiao Chen +5 more

cs.CV · June 2025 · cited 0× · arXiv ↗ · more like this ·

Class-incremental learning (CIL) enables models to learn new classes continually without forgetting previously acquired knowledge. Multi-label CIL (MLCIL) extends CIL to a real-world scenario where each sample may belong to multiple classes, introducing several challenges: label absence, which leads to incomplete historical information due to missing labels, and class imbalance, which results in the model bias toward majority classes. To address these challenges, we propose Label-Augmented Analytic Adaptation (L3A), an exemplar-free approach without storing past samples. L3A integrates two key modules. The pseudo-label (PL) module implements label augmentation by generating pseudo-labels for current phase samples, addressing the label absence problem. The weighted analytic classifier (WAC) derives a closed-form solution for neural networks. It introduces sample-specific weights to…

AFCL: Analytic Federated Continual Learning for Spatio-Temporal Invariance of Non-IID Data

Jianheng Tang, Huiping Zhuang, Jingyu He +10 more

cs.LG · 18 May 2025 · cited 0× · arXiv ↗ · more like this ·

Federated Continual Learning (FCL) enables distributed clients to collaboratively train a global model from online task streams in dynamic real-world scenarios. However, existing FCL methods face challenges of both spatial data heterogeneity among distributed clients and temporal data heterogeneity across online tasks. Such data heterogeneity significantly degrades the model performance with severe spatial-temporal catastrophic forgetting of local and past knowledge. In this paper, we identify that the root cause of this issue lies in the inherent vulnerability and sensitivity of gradients to non-IID data. To fundamentally address this issue, we propose a gradient-free method, named Analytic Federated Continual Learning (AFCL), by deriving analytical (i.e., closed-form) solutions from frozen extracted features. In local training, our AFCL enables single-epoch learning with only a…

Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Class-Incremental Learning

Run He, Di Fang, Yicheng Xu +5 more

cs.CV · March 2025 · cited 0× · arXiv ↗ · more like this ·

Exemplar-Free Class-Incremental Learning (EFCIL) aims to sequentially learn from distinct categories without retaining exemplars but easily suffers from catastrophic forgetting of learned knowledge. While existing EFCIL methods leverage knowledge distillation to alleviate forgetting, they still face two critical challenges: semantic shift and decision bias. Specifically, the embeddings of old tasks shift in the embedding space after learning new tasks, and the classifier becomes biased towards new tasks due to training solely with new data, hindering the balance between old and new knowledge. To address these issues, we propose the Dual-Projection Shift Estimation and Classifier Reconstruction (DPCR) approach for EFCIL. DPCR effectively estimates semantic shift through a dual-projection, which combines a learnable transformation with a row-space projection to capture both task-wise and…

Analytic Subspace Routing: How Recursive Least Squares Works in Continual Learning of Large Language Model

Kai Tong, Kang Pan, Xiao Zhang +5 more

cs.LG · March 2025 · cited 0× · arXiv ↗ · more like this ·

Large Language Models (LLMs) possess encompassing capabilities that can process diverse language-related tasks. However, finetuning on LLMs will diminish this general skills and continual finetuning will further cause severe degradation on accumulated knowledge. Recently, Continual Learning (CL) in Large Language Models (LLMs) arises which aims to continually adapt the LLMs to new tasks while maintaining previously learned knowledge and inheriting general skills. Existing techniques either leverage previous data to replay, leading to extra computational costs, or utilize a single parameter-efficient module to learn the downstream task, constraining new knowledge absorption with interference between different tasks. Toward these issues, this paper proposes Analytic Subspace Routing(ASR) to address these challenges. For each task, we isolate the learning within a subspace of deep layers'…

GACL: Exemplar-Free Generalized Analytic Continual Learning

Huiping Zhuang, Yizhu Chen, Di Fang +5 more

cs.LG · March 2024 · cited 0× · arXiv ↗ · more like this ·

Class incremental learning (CIL) trains a network on sequential tasks with separated categories in each task but suffers from catastrophic forgetting, where models quickly lose previously learned knowledge when acquiring new tasks. The generalized CIL (GCIL) aims to address the CIL problem in a more real-world scenario, where incoming data have mixed data categories and unknown sample size distribution. Existing attempts for the GCIL either have poor performance or invade data privacy by saving exemplars. In this paper, we propose a new exemplar-free GCIL technique named generalized analytic continual learning (GACL). The GACL adopts analytic learning (a gradient-free training technique) and delivers an analytical (i.e., closed-form) solution to the GCIL scenario. This solution is derived via decomposing the incoming data into exposed and unexposed classes, thereby attaining a…

F-OAL: Forward-only Online Analytic Learning with Fast Training and Low Memory Footprint in Class Incremental Learning

Huiping Zhuang, Yuchen Liu, Run He +5 more

cs.CV · March 2024 · cited 0× · arXiv ↗ · more like this ·

Online Class Incremental Learning (OCIL) aims to train models incrementally, where data arrive in mini-batches, and previous data are not accessible. A major challenge in OCIL is Catastrophic Forgetting, i.e., the loss of previously learned knowledge. Among existing baselines, replay-based methods show competitive results but requires extra memory for storing exemplars, while exemplar-free (i.e., data need not be stored for replay in production) methods are resource-friendly but often lack accuracy. In this paper, we propose an exemplar-free approach--Forward-only Online Analytic Learning (F-OAL). Unlike traditional methods, F-OAL does not rely on back-propagation and is forward-only, significantly reducing memory usage and computational time. Cooperating with a pre-trained frozen encoder with Feature Fusion, F-OAL only needs to update a linear classifier by recursive least square. This…

REAL: Representation Enhanced Analytic Learning for Exemplar-free Class-incremental Learning

Run He, Di Fang, Yizhu Chen +5 more

cs.LG · March 2024 · cited 0× · arXiv ↗ · more like this ·

Exemplar-free class-incremental learning (EFCIL) aims to mitigate catastrophic forgetting in class-incremental learning (CIL) without available historical training samples as exemplars. Compared with its exemplar-based CIL counterpart that stores exemplars, EFCIL suffers more from forgetting issues. Recently, a new EFCIL branch named Analytic Continual Learning (ACL) introduces a gradient-free paradigm via Recursive Least-Square, achieving a forgetting-resistant classifier training with a frozen backbone during CIL. However, existing ACL suffers from ineffective representations and insufficient utilization of backbone knowledge. In this paper, we propose a representation-enhanced analytic learning (REAL) to address these problems. To enhance the representation, REAL constructs a dual-stream base pretraining followed by representation enhancing distillation process. The dual-stream base…

Local application-oriented system design to extract metadata from sub-image of MODIS

Bai Kai-xu, Shi Run-he, Chaoshun Liu +2 more

Artificial Intelligence · October 2010 · cited 0× · DOI ↗ · more like this ·

Remote sensing technique has become the main method for earth observation in the 21st century, and it has been widely used in environmental disaster monitoring and resources survey at regional and global scales. In recent years, remote sensing data type enriches constantly and presents to be mass data and multi-source increasingly. However, the utilization of remote sensing data remains at a low level, especially for moderate resolution images, which cover a large area and are valuable in multi-discipline applications. One reason is that the MODIS data are inconvenience for users in local remote sensing applications. In this paper, we introduced a local application-oriented system, and extracted metadata of each adminastritive region from MODIS. The system aims at extracting metadata which contains attribute information about administrative unit from local sub-image, and the system will…

Electrophotocatalytic\nThiocyanation and Sulfonylation\nCyclization of Unactivated Alkenes

Yingchun Ma (9703385), Ping Yu (215536), Ruoyu Qin (20449649) +5 more

Transportation · Figshare · 1753 · cited 0× · DOI ↗ · more like this ·

We report an electrophotocatalytic process that enables\nthe thiocyanation\nand sulfonylation/cyclization of alkenes. It is applicable to a wide\nrange of unactivated alkenes, using the inexpensive photocatalyst\n2,4,6-triphenylpyrylium tetrafluoroborate (TPPT) to produce a diverse\narray of heterocycles containing sulfonyl and thiocyano groups with\ngood functional group tolerance. The protocol operates under mild,\nchemical oxidant- and transition-metal-free, with a broad scope of\nsubstrates. Preliminary mechanistic studies suggest that the reaction\ninvolves a combination of electrolysis and the reductive quenching\nphotocatalytic cycle of TPPT.

Non-Forgetting Knowledge Allocation with Bi-level Competition for Class-Incremental Learning

Xiang Tan, Run He, Yawen Cui +6 more

cs.CV · 28 May 2026 · arXiv ↗ · more like this ·

Class-Incremental Learning (CIL) with pre-trained models (PTMs) aims to sequentially adapt PTMs to new categories without forgetting old knowledge. Built upon PTMs, existing adapter-based methods mainly train models via distinct task-specific adapters, and present a uniform knowledge allocation for each adapter during inference. However, this allocation mechanism ignores the nature of task discrepancy and leads to suboptimal utilization of adapters. Also, under CIL constraint, an allocator is prone to forgetting when tasks evolve. To address these issues, we propose a Non-Forgetting Allocation with Bi-Level Competition (NoFA-BC). NoFA-BC constructs a non-forgetting allocator (NFA) by transforming the allocator training into a recursive least-squares problem and achieves an allocator equivalent to that trained with all data. Based on the NFA, a Bi-Level Competition (BLC) including an…

Rethinking Adapter Placement: A Dominant Adaptation Module Perspective

Suoxin Zhang, Run He, Di Fang +3 more

cs.AI · 7 May 2026 · arXiv ↗ · more like this ·

Low-rank adaptation (LoRA) is a widely used parameter-efficient fine-tuning method that places trainable low-rank adapters into frozen pre-trained models. Recent studies show that using fewer LoRA adapters may still maintain or even improve performance, but existing methods still distribute adapters broadly, leaving \emph{where to place a limited number of adapters to maximize performance} largely open. To investigate this, we introduce \textbf{PAGE} (\textbf{P}rojected \textbf{A}dapter \textbf{G}radient \textbf{E}nergy), a gradient-based sensitivity probe that estimates the initial trainable gradient energy available to each candidate LoRA adapter. Surprisingly, we find that PAGE is highly concentrated on a single shallow FFN down-projection across two model families and four downstream tasks. We term this module the \textbf{dominant adaptation module} and show that its layer index is…

DeepAFL: Deep Analytic Federated Learning

Jianheng Tang, Yajiang Huang, Kejia Fan +8 more

cs.LG · 28 Feb 2026 · arXiv ↗ · more like this ·

Federated Learning (FL) is a popular distributed learning paradigm to break down data silo. Traditional FL approaches largely rely on gradient-based updates, facing significant issues about heterogeneity, scalability, convergence, and overhead, etc. Recently, some analytic-learning-based work has attempted to handle these issues by eliminating gradient-based updates via analytical (i.e., closed-form) solutions. Despite achieving superior invariance to data heterogeneity, these approaches are fundamentally limited by their single-layer linear model with a frozen pre-trained backbone. As a result, they can only achieve suboptimal performance due to their lack of representation learning capabilities. In this paper, to enable representable analytic models while preserving the ideal invariance to data heterogeneity for FL, we propose our Deep Analytic Federated Learning approach, named…