arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Shaobo Lin

30 papers · 2009–2025

2009: 12010: 12011: 22012: 02013: 52014: 52015: 52016: 32017: 02018: 22019: 02020: 02021: 02022: 12023: 32024: 02025: 2
2009peak 5 in 20132025

Frequent co-authors

Papers

showing 1–25 of 30 RSS

Is Extreme Learning Machine Feasible? A Theoretical Assessment (Part II)

Shaobo Lin, Xia Liu, Jian Fang +1 more

cs.LG · · cited 121× · arXiv ↗ · more like this ·

An extreme learning machine (ELM) can be regarded as a two stage feed-forward neural network (FNN) learning system which randomly assigns the connections with and within hidden neurons in the first stage and tunes the connections with output neurons in the second stage. Therefore, ELM training is essentially a linear learning problem, which significantly reduces the computational burden. Numerous applications show that such a computation burden reduction does not degrade the generalization capability. It has, however, been open that whether this is true in theory. The aim of our work is to study the theoretical feasibility of ELM by analyzing the pros and cons of ELM. In the previous part on this topic, we pointed out that via appropriate selection of the activation function, ELM does not degrade the generalization capability in the expectation sense. In this paper, we launch the study…

Global Convergence of Block Coordinate Descent in Deep Learning

Jinshan Zeng, Tim Tsz-Kit Lau, Shaobo Lin +1 more

math.OC · · cited 103× · arXiv ↗ · more like this ·

Deep learning has aroused extensive attention due to its great empirical success. The efficiency of the block coordinate descent (BCD) methods has been recently demonstrated in deep neural network (DNN) training. However, theoretical studies on their convergence properties are limited due to the highly nonconvex nature of DNN training. In this paper, we aim at providing a general methodology for provable convergence guarantees for this type of methods. In particular, for most of the commonly used DNN training models involving both two- and three-splitting schemes, we establish the global convergence to a critical point at a rate of ${\cal O}(1/k)$, where $k$ is the number of iterations. The results extend to general loss functions which have Lipschitz continuous gradients and deep residual networks (ResNets). Our key development adds several new elements to the Kurdyka-Łojasiewicz…

Explore the Power of Synthetic Data on Few-shot Object Detection

Shaobo Lin, Kun Wang, Xingyu Zeng +1 more

cs.CV · · cited 67× · arXiv ↗ · more like this ·

Few-shot object detection (FSOD) aims to expand an object detector for novel categories given only a few instances for training. The few training samples restrict the performance of FSOD model. Recent text-to-image generation models have shown promising results in generating high-quality images. How applicable these synthetic images are for FSOD tasks remains under-explored. This work extensively studies how synthetic images generated from state-of-the-art text-to-image generators benefit FSOD tasks. We focus on two perspectives: (1) How to use synthetic data for FSOD? (2) How to find representative samples from the large-scale synthetic dataset? We design a copy-paste-based pipeline for using synthetic data. Specifically, saliency object detection is applied to the original generated image, and the minimum enclosing box is used for cropping the main object based on the saliency map.…

Divide and Conquer Local Average Regression

Xiangyu Chang, Shaobo Lin, Yao Wang

cs.LG · · cited 42× · arXiv ↗ · more like this ·

The divide and conquer strategy, which breaks a massive data set into a se- ries of manageable data blocks, and then combines the independent results of data blocks to obtain a final decision, has been recognized as a state-of-the-art method to overcome challenges of massive data analysis. In this paper, we merge the divide and conquer strategy with local average regression methods to infer the regressive relationship of input-output pairs from a massive data set. After theoretically analyzing the pros and cons, we find that although the divide and conquer local average regression can reach the optimal learning rate, the restric- tion to the number of data blocks is a bit strong, which makes it only feasible for small number of data blocks. We then propose two variants to lessen (or remove) this restriction. Our results show that these variants can achieve the optimal learning rate with…

Constructive neural network learning

Shaobo Lin, Jinshan Zeng, Xiaoqin Zhang

cs.LG · · cited 31× · arXiv ↗ · more like this ·

In this paper, we aim at developing scalable neural network-type learning systems. Motivated by the idea of "constructive neural networks" in approximation theory, we focus on "constructing" rather than "training" feed-forward neural networks (FNNs) for learning, and propose a novel FNNs learning system called the constructive feed-forward neural network (CFN). Theoretically, we prove that the proposed method not only overcomes the classical saturation problem for FNN approximation, but also reaches the optimal learning rate when the regression function is smooth, while the state-of-the-art learning rates established for traditional FNNs are only near optimal (up to a logarithmic factor). A series of numerical simulations are provided to show the efficiency and feasibility of CFN via comparing with the well-known regularized least squares (RLS) with Gaussian kernel and extreme learning…

Learning Capability of Relaxed Greedy Algorithms

Shaobo Lin, Yuanhua Rong, Xingping Sun +1 more

Artificial Intelligence · IEEE Transactions on Neural Networks and Learning Systems · · cited 29× · DOI ↗ · more like this ·

In the practice of machine learning, one often encounters problems in which noisy data are abundant while the learning targets are imprecise and elusive. To these challenges, most of the traditional learning algorithms employ hypothesis spaces of large capacity. This has inevitably led to high computational burdens and caused considerable machine sluggishness. Utilizing greedy algorithms in this kind of learning environment has greatly improved machine performance. The best existing learning rate of various greedy algorithms is proved to achieve the order of (m/log m)(-1/2), where m is the sample size. In this paper, we provide a relaxed greedy algorithm and study its learning capability. We prove that the learning rate of the new relaxed greedy algorithm is faster than the order m(-1/2). Unlike many other greedy algorithms, which are often indecisive issuing a stopping order to the…

An Effective Crop-Paste Pipeline for Few-shot Object Detection

Shaobo Lin, Kun Wang, Xingyu Zeng +1 more

cs.CV · · cited 11× · arXiv ↗ · more like this ·

Few-shot object detection (FSOD) aims to expand an object detector for novel categories given only a few instances for training. However, detecting novel categories with only a few samples usually leads to the problem of misclassification. In FSOD, we notice the false positive (FP) of novel categories is prominent, in which the base categories are often recognized as novel ones. To address this issue, a novel data augmentation pipeline that Crops the Novel instances and Pastes them on the selected Base images, called CNPB, is proposed. There are two key questions to be answered: (1) How to select useful base images? and (2) How to combine novel and base data? We design a multi-step selection strategy to find useful base data. Specifically, we first discover the base images which contain the FP of novel categories and select a certain amount of samples from them for the base and novel…

Learning through deterministic assignment of hidden parameters

Jian Fang, Shaobo Lin, Zongben Xu

cs.LG · · cited 9× · arXiv ↗ · more like this ·

Supervised learning frequently boils down to determining hidden and bright parameters in a parameterized hypothesis space based on finite input-output samples. The hidden parameters determine the attributions of hidden predictors or the nonlinear mechanism of an estimator, while the bright parameters characterize how hidden predictors are linearly combined or the linear mechanism. In traditional learning paradigm, hidden and bright parameters are not distinguished and trained simultaneously in one learning process. Such an one-stage learning (OSL) brings a benefit of theoretical analysis but suffers from the high computational burden. To overcome this difficulty, a two-stage learning (TSL) scheme, featured by learning through deterministic assignment of hidden parameters (LtDaHP) was proposed, which suggests to deterministically generate the hidden parameters by using minimal Riesz…

Re-scale boosting for regression and classification

Shaobo Lin, Yao Wang, Lin Xu

cs.LG · · cited 5× · arXiv ↗ · more like this ·

Boosting is a learning scheme that combines weak prediction rules to produce a strong composite estimator, with the underlying intuition that one can obtain accurate prediction rules by combining "rough" ones. Although boosting is proved to be consistent and overfitting-resistant, its numerical convergence rate is relatively slow. The aim of this paper is to develop a new boosting strategy, called the re-scale boosting (RBoosting), to accelerate the numerical convergence rate and, consequently, improve the learning performance of boosting. Our studies show that RBoosting possesses the almost optimal numerical convergence rate in the sense that, up to a logarithmic factor, it can reach the minimax nonlinear approximation rate. We then use RBoosting to tackle both the classification and regression problems, and deduce a tight generalization error estimate. The theoretical and experimental…

The estimate for approximation error of spherical neural networks

Feilong Cao, Huazhong Wang, Shaobo Lin

Artificial Intelligence · Mathematical Methods in the Applied Sciences · · cited 4× · DOI ↗ · more like this ·

Compared with planar hyperplane, fitting data on the sphere has been an important and an active issue in geoscience, metrology, brain imaging, and so on. In this paper, with the help of the Jackson-type theorem of polynomial approximation on the sphere, we construct spherical feed-forward neural networks to approximate the continuous function defined on the sphere. As a metric, the modulus of smoothness of spherical function is used to measure the error of the approximation, and a Jackson-type theorem on the approximation is established. Copyright © 2011 John Wiley & Sons, Ltd.

Widening income inequality and the decline in fertility intentions: Micro evidence from Chinese households

Shaobo Lin, Nengsheng Luo

Demography · International Review of Economics & Finance · · cited 1× · DOI ↗ · more like this ·

The persistent decline in global fertility rates poses a critical demographic challenge, and widening income inequality is increasingly recognized as a structural impediment to reproductive decision-making. Drawing on nationally representative micro-level panel data from China, this study examines the causal effect of household income inequality on fertility intentions and the mechanisms underlying this relationship. Using a rigorous identification strategy, we find that broader inter-household disparities significantly reduce fertility intentions; the results are robust to extensive checks and endogeneity corrections. Heterogeneity analyses show stronger effects among households with older average spousal age, prior fertility experience, and lower wealth. Mechanism tests identify rising child-rearing costs and declining expectations of upward mobility as key channels linking inequality…

L~p Approximation by Shifts of Smooth Kernels on Spheres

Shaobo Lin

Numerical Analysis · Chinese Annals of Mathematics · · cited 1× · OpenAlex ↗ · more like this ·

The authors embed the smooth radial basis functions in a larger native space generated by a less smooth kernel,and use them,in L~p metric,to approximate functions from the larger native space on the unit sphere.As a result,the L~p error bound between the best approximant and the target function is established.

Markov Inequalities for Polynomials with Restricted Coefficients

Feilong Cao, Shaobo Lin

Applied Mathematics · Journal of Inequalities and Applications · · cited 1× · DOI ↗ · more like this ·

<p/> Essentially sharp Markov-type inequalities are known for various classes of polynomials with constraints including constraints of the coefficients of the polynomials. For <inline-formula> <graphic file="1029-242X-2009-808720-i1.gif"/></inline-formula> and <inline-formula> <graphic file="1029-242X-2009-808720-i2.gif"/></inline-formula> we introduce the class <inline-formula> <graphic file="1029-242X-2009-808720-i3.gif"/></inline-formula> as the collection of all polynomials of the form <inline-formula> <graphic file="1029-242X-2009-808720-i4.gif"/></inline-formula>, <inline-formula> <graphic file="1029-242X-2009-808720-i5.gif"/></inline-formula>, <inline-formula> <graphic file="1029-242X-2009-808720-i6.gif"/></inline-formula>, <inline-formula> <graphic file="1029-242X-2009-808720-i7.gif"/></inline-formula>. In this paper, we prove essentially sharp Markov-type inequalities for…

Unveiling Lagrange’s Theorem and its Applications in Proving other Theorems

Shaobo Lin

Geometry and Topology · Highlights in Science Engineering and Technology · · cited 0× · DOI ↗ · more like this ·

The group theory is an essential subject in Mathematics and Physics, and the target of this paper is to prove a famous theorem in group theory, so-called Lagrange’s Theorem. By using some really basic definitions of group, the exsistance of this theorem is essential for abstract algebra. In this paper, the author will focus on how to prove Lagrange’s Theorem step by step from the base of group theorey, mainly by using the nature of cosets and how does each coset in the same subgroup behaves to get the final result. Ultimately, the author will demonstrate that the order of the group is divisible by the order of its subset. As mentioned ealier, the status of this theorem is unshakable. Because of this theorem, many other corollary theorem was discovered, for example Wilson’s Theorem and etc. All of these corollaries are very important in modern technologys, going deep into this theorem…

Explore the Power of Dropout on Few-shot Learning

Shaobo Lin, Xingyu Zeng, Rui Zhao

cs.CV · · cited 0× · arXiv ↗ · more like this ·

The generalization power of the pre-trained model is the key for few-shot deep learning. Dropout is a regularization technique used in traditional deep learning methods. In this paper, we explore the power of dropout on few-shot learning and provide some insights about how to use it. Extensive experiments on the few-shot object detection and few-shot image classification datasets, i.e., Pascal VOC, MS COCO, CUB, and mini-ImageNet, validate the effectiveness of our method.

A Unified Framework with Meta-dropout for Few-shot Learning

Shaobo Lin, Xingyu Zeng, Rui Zhao

cs.CV · · cited 0× · arXiv ↗ · more like this ·

Conventional training of deep neural networks usually requires a substantial amount of data with expensive human annotations. In this paper, we utilize the idea of meta-learning to explain two very different streams of few-shot learning, i.e., the episodic meta-learning-based and pre-train finetune-based few-shot learning, and form a unified meta-learning framework. In order to improve the generalization power of our framework, we propose a simple yet effective strategy named meta-dropout, which is applied to the transferable knowledge generalized from base categories to novel categories. The proposed strategy can effectively prevent neural units from co-adapting excessively in the meta-training stage. Extensive experiments on the few-shot object detection and few-shot image classification datasets, i.e., Pascal VOC, MS COCO, CUB, and mini-ImageNet, validate the effectiveness of our…

Model selection of polynomial kernel regression

Shaobo Lin, Xingping Sun, Zongben Xu +1 more

cs.LG · · cited 0× · arXiv ↗ · more like this ·

Polynomial kernel regression is one of the standard and state-of-the-art learning strategies. However, as is well known, the choices of the degree of polynomial kernel and the regularization parameter are still open in the realm of model selection. The first aim of this paper is to develop a strategy to select these parameters. On one hand, based on the worst-case learning rate analysis, we show that the regularization term in polynomial kernel regression is not necessary. In other words, the regularization parameter can decrease arbitrarily fast when the degree of the polynomial kernel is suitable tuned. On the other hand,taking account of the implementation of the algorithm, the regularization term is required. Summarily, the effect of the regularization term in polynomial kernel regression is only to circumvent the " ill-condition" of the kernel matrix. Based on this, the second…

Sparse Solution of Underdetermined Linear Equations via Adaptively Iterative Thresholding

Jinshan Zeng, Shaobo Lin, Zongben Xu

cs.IT · · cited 0× · arXiv ↗ · more like this ·

Finding the sparset solution of an underdetermined system of linear equations $y=Ax$ has attracted considerable attention in recent years. Among a large number of algorithms, iterative thresholding algorithms are recognized as one of the most efficient and important classes of algorithms. This is mainly due to their low computational complexities, especially for large scale applications. The aim of this paper is to provide guarantees on the global convergence of a wide class of iterative thresholding algorithms. Since the thresholds of the considered algorithms are set adaptively at each iteration, we call them adaptively iterative thresholding (AIT) algorithms. As the main result, we show that as long as $A$ satisfies a certain coherence property, AIT algorithms can find the correct support set within finite iterations, and then converge to the original sparse solution exponentially…

Strong Converse Inequality for a Spherical Operator

Shaobo Lin, Feilong Cao

Mathematical Physics · Journal of Inequalities and Applications · · cited 0× · DOI ↗ · more like this ·

In the paper titled as "Jackson-type inequality on the sphere" (2004), Ditzian introduced a spherical nonconvolution operator , which played an important role in the proof of the well-known Jackson inequality for spherical harmonics. In this paper, we give the lower bound of approximation by this operator. Namely, we prove that there are constants and such that for any th Lebesgue integrable or continuous function defined on the sphere, where is the th modulus of smoothness of .

Greedy Criterion in Orthogonal Greedy Learning

Lin Xu, Shaobo Lin, Jinshan Zeng +2 more

cs.LG · · arXiv ↗ · more like this ·

Orthogonal greedy learning (OGL) is a stepwise learning scheme that starts with selecting a new atom from a specified dictionary via the steepest gradient descent (SGD) and then builds the estimator through orthogonal projection. In this paper, we find that SGD is not the unique greedy criterion and introduce a new greedy criterion, called "$δ$-greedy threshold" for learning. Based on the new greedy criterion, we derive an adaptive termination rule for OGL. Our theoretical study shows that the new learning scheme can achieve the existing (almost) optimal learning rate of OGL. Plenty of numerical experiments are provided to support that the new scheme can achieve almost optimal generalization performance, while requiring less computation than OGL.

A Gauss-Seidel Iterative Thresholding Algorithm for lq Regularized Least Squares Regression

Jinshan Zeng, Zhimin Peng, Shaobo Lin

math.NA · · arXiv ↗ · more like this ·

In recent studies on sparse modeling, $l_q$ ($0<q<1$) regularized least squares regression ($l_q$LS) has received considerable attention due to its superiorities on sparsity-inducing and bias-reduction over the convex counterparts. In this paper, we propose a Gauss-Seidel iterative thresholding algorithm (called GAITA) for solution to this problem. Different from the classical iterative thresholding algorithms using the Jacobi updating rule, GAITA takes advantage of the Gauss-Seidel rule to update the coordinate coefficients. Under a mild condition, we can justify that the support set and sign of an arbitrary sequence generated by GAITA will converge within finite iterations. This convergence property together with the Kurdyka-Łojasiewicz property of ($l_q$LS) naturally yields the strong convergence of GAITA under the same condition as above, which is generally weaker than the condition…

Shrinkage degree in $L_2$-re-scale boosting for regression

Lin Xu, Shaobo Lin, Yao Wang +1 more

cs.LG · · arXiv ↗ · more like this ·

Re-scale boosting (RBoosting) is a variant of boosting which can essentially improve the generalization performance of boosting learning. The key feature of RBoosting lies in introducing a shrinkage degree to re-scale the ensemble estimate in each gradient-descent step. Thus, the shrinkage degree determines the performance of RBoosting. The aim of this paper is to develop a concrete analysis concerning how to determine the shrinkage degree in $L_2$-RBoosting. We propose two feasible ways to select the shrinkage degree. The first one is to parameterize the shrinkage degree and the other one is to develope a data-driven approach of it. After rigorously analyzing the importance of the shrinkage degree in $L_2$-RBoosting learning, we compare the pros and cons of the proposed methods. We find that although these approaches can reach the same learning rates, the structure of the final…

Nonparametric regression using needlet kernels for spherical data

Shaobo Lin

cs.LG · · arXiv ↗ · more like this ·

Needlets have been recognized as state-of-the-art tools to tackle spherical data, due to their excellent localization properties in both spacial and frequency domains. This paper considers developing kernel methods associated with the needlet kernel for nonparametric regression problems whose predictor variables are defined on a sphere. Due to the localization property in the frequency domain, we prove that the regularization parameter of the kernel ridge regression associated with the needlet kernel can decrease arbitrarily fast. A natural consequence is that the regularization term for the kernel ridge regression is not necessary in the sense of rate optimality. Based on the excellent localization property in the spacial domain further, we also prove that all the $l^{q}$ $(01\leq q < \infty)$ kernel regularization estimates associated with the needlet kernel, including the kernel…

Greedy metrics in orthogonal greedy learning

Lin Xu, Shaobo Lin, Jinshan Zeng +1 more

cs.LG · · arXiv ↗ · more like this ·

Orthogonal greedy learning (OGL) is a stepwise learning scheme that adds a new atom from a dictionary via the steepest gradient descent and build the estimator via orthogonal projecting the target function to the space spanned by the selected atoms in each greedy step. Here, "greed" means choosing a new atom according to the steepest gradient descent principle. OGL then avoids the overfitting/underfitting by selecting an appropriate iteration number. In this paper, we point out that the overfitting/underfitting can also be avoided via redefining "greed" in OGL. To this end, we introduce a new greedy metric, called $δ$-greedy thresholds, to refine "greed" and theoretically verifies its feasibility. Furthermore, we reveals that such a greedy metric can bring an adaptive termination rule on the premise of maintaining the prominent learning performance of OGL. Our results show that the…

Learning and approximation capability of orthogonal super greedy algorithm

Jian Fang, Shaobo Lin, Zongben Xu

cs.LG · · arXiv ↗ · more like this ·

We consider the approximation capability of orthogonal super greedy algorithms (OSGA) and its applications in supervised learning. OSGA is concerned with selecting more than one atoms in each iteration step, which, of course, greatly reduces the computational burden when compared with the conventional orthogonal greedy algorithm (OGA). We prove that even for function classes that are not the convex hull of the dictionary, OSGA does not degrade the approximation capability of OGA provided the dictionary is incoherent. Based on this, we deduce a tight generalization error bound for OSGA learning. Our results show that in the realm of supervised learning, OSGA provides a possibility to further reduce the computational burden of OGA in the premise of maintaining its prominent generalization capability.