Author

Lei Yu

1,129 papers · 1753–2026

1967peak 108 in 20242026

Frequent co-authors

Papers

showing 1–25 of 1,129 RSS

Toward integrating feature selection algorithms for classification and clustering

Information Systems · IEEE Transactions on Knowledge and Data Engineering · 7 Mar 2005 · cited 2,680× · DOI ↗ · more like this ·

This paper introduces concepts and algorithms of feature selection, surveys existing feature selection algorithms for classification and clustering, groups and compares different algorithms with a categorizing framework based on search strategies, evaluation criteria, and data mining tasks, reveals unattempted combinations, and provides guidelines in selecting feature selection algorithms. With the categorizing framework, we continue our efforts toward-building an integrated system for intelligent feature selection. A unifying platform is proposed as an intermediate step. An illustrative example is presented to show how existing feature selection algorithms can be integrated into a meta algorithm that can take advantage of individual algorithms. An added advantage of doing so is to help a user employ a suitable algorithm without knowing details of each algorithm. Some real-world…

Feature selection for high-dimensional data: a fast correlation-based filter solution

Lei Yu, Huan Liu

Computer Vision and Pattern Recognition · International Conference on Machine Learning · 21 Aug 2003 · cited 2,213× · OpenAlex ↗ · more like this ·

Feature selection, as a preprocessing step to machine learning, has been effective in reducing dimensionality, removing irrelevant data, increasing learning accuracy, and improving comprehensibility. However, the recent increase of dimensionality of data poses a severe challenge to many existing feature selection methods with respect to efficiency and effectiveness. In this work, we introduce a novel concept, predominant correlation, and propose a fast filter method which can identify relevant features as well as redundancy among relevant features without pairwise correlation analysis. The efficiency and effectiveness of our method is demonstrated through extensive comparisons with other methods using real-world data of high dimensionality. 1.

Efficient Feature Selection via Analysis of Relevance and Redundancy

Lei Yu, Huan Liu

Computer Vision and Pattern Recognition · Journal of Machine Learning Research · December 2004 · cited 1,978× · OpenAlex ↗ · more like this ·

Feature selection is applied to reduce the number of features in many applications where data has hundreds or thousands of features. Existing feature selection methods mainly focus on finding relevant features. In this paper, we show that feature relevance alone is insufficient for efficient feature selection of high-dimensional data. We define feature redundancy and propose to perform explicit redundancy analysis in feature selection. A new framework is introduced that decouples relevance analysis and redundancy analysis. We develop a correlation-based method for relevance and redundancy analysis, and conduct an empirical study of its efficiency and effectiveness comparing with representative methods.

Neural Variational Inference for Text Processing

Yishu Miao, Lei Yu, Phil Blunsom

cs.CL · November 2015 · cited 687× · arXiv ↗ · more like this ·

Recent advances in neural variational inference have spawned a renaissance in deep latent variable models. In this paper we introduce a generic variational inference framework for generative and conditional models of text. While traditional variational methods derive an analytic approximation for the intractable distributions over latent variables, here we construct an inference network conditioned on the discrete text input to provide the variational distribution. We validate this framework on two very different text modelling applications, generative document modelling and supervised question answering. Our neural variational document model combines a continuous stochastic document representation with a bag-of-words generative model and achieves the lowest reported perplexities on two standard test corpora. The neural answer selection model employs a stochastic representation layer…

A Normalized Gaussian Wasserstein Distance for Tiny Object Detection

Jinwang Wang, Chang Xu, Wen Yang +1 more

cs.CV · October 2021 · cited 643× · arXiv ↗ · more like this ·

Detecting tiny objects is a very challenging problem since a tiny object only contains a few pixels in size. We demonstrate that state-of-the-art detectors do not produce satisfactory results on tiny objects due to the lack of appearance information. Our key observation is that Intersection over Union (IoU) based metrics such as IoU itself and its extensions are very sensitive to the location deviation of the tiny objects, and drastically deteriorate the detection performance when used in anchor-based detectors. To alleviate this, we propose a new evaluation metric using Wasserstein distance for tiny object detection. Specifically, we first model the bounding boxes as 2D Gaussian distributions and then propose a new metric dubbed Normalized Wasserstein Distance (NWD) to compute the similarity between them by their corresponding Gaussian distributions. The proposed NWD metric can be…

Deep Learning for Answer Sentence Selection

Lei Yu, Karl Moritz Hermann, Phil Blunsom +1 more

cs.CL · 4 Dec 2014 · cited 422× · arXiv ↗ · more like this ·

Answer sentence selection is the task of identifying sentences that contain the answer to a given question. This is an important problem in its own right as well as in the larger context of open domain question answering. We propose a novel approach to solving this task via means of distributed representations, and learn to match questions with answers by considering their semantic encoding. This contrasts prior work on this task, which typically relies on classifiers with large numbers of hand-crafted syntactic and semantic features and various external resources. Our approach does not require any feature engineering nor does it involve specialist linguistic data, making this model easily applicable to a wide range of domains and languages. Experimental results on a standard benchmark dataset from TREC demonstrate that---despite its simplicity---our model matches state of the art…

RFLA: Gaussian Receptive Field based Label Assignment for Tiny Object Detection

Chang Xu, Jinwang Wang, Wen Yang +3 more

cs.CV · August 2022 · cited 312× · arXiv ↗ · more like this ·

Detecting tiny objects is one of the main obstacles hindering the development of object detection. The performance of generic object detectors tends to drastically deteriorate on tiny object detection tasks. In this paper, we point out that either box prior in the anchor-based detector or point prior in the anchor-free detector is sub-optimal for tiny objects. Our key observation is that the current anchor-based or anchor-free label assignment paradigms will incur many outlier tiny-sized ground truth samples, leading to detectors imposing less focus on the tiny objects. To this end, we propose a Gaussian Receptive Field based Label Assignment (RFLA) strategy for tiny object detection. Specifically, RFLA first utilizes the prior information that the feature receptive field follows Gaussian distribution. Then, instead of assigning samples with IoU or center sampling strategy, a new…

Differentially Private Model Publishing for Deep Learning

Lei Yu, Ling Liu, Calton Pu +2 more

cs.CR · April 2019 · cited 296× · arXiv ↗ · more like this ·

Deep learning techniques based on neural networks have shown significant success in a wide range of AI tasks. Large-scale training datasets are one of the critical factors for their success. However, when the training datasets are crowdsourced from individuals and contain sensitive information, the model parameters may encode private information and bear the risks of privacy leakage. The recent growing trend of the sharing and publishing of pre-trained models further aggravates such privacy risks. To tackle this problem, we propose a differentially private approach for training neural networks. Our approach includes several new techniques for optimizing both privacy loss and model accuracy. We employ a generalization of differential privacy called concentrated differential privacy(CDP), with both a formal and refined privacy loss analysis on two different data batching methods. We…

BiFA-YOLO: A Novel YOLO-Based Method for Arbitrary-Oriented Ship Detection in High-Resolution SAR Images

Zhongzhen Sun, Xiangguang Leng, Lei Yu +3 more

Computer Vision and Pattern Recognition · Remote Sensing · 20 Oct 2021 · cited 278× · DOI ↗ · more like this ·

Due to its great application value in the military and civilian fields, ship detection in synthetic aperture radar (SAR) images has always attracted much attention. However, ship targets in High-Resolution (HR) SAR images show the significant characteristics of multi-scale, arbitrary directions and dense arrangement, posing enormous challenges to detect ships quickly and accurately. To address these issues above, a novel YOLO-based arbitrary-oriented SAR ship detector using bi-directional feature fusion and angular classification (BiFA-YOLO) is proposed in this article. First of all, a novel bi-directional feature fusion module (Bi-DFFM) tailored to SAR ship detection is applied to the YOLO framework. This module can efficiently aggregate multi-scale features through bi-directional (top-down and bottom-up) information interaction, which is helpful for detecting multi-scale ships.…

Demystifying Membership Inference Attacks in Machine Learning as a Service

Stacey Truex, Ling Liu, Mehmet Emre Gürsoy +2 more

Artificial Intelligence · IEEE Transactions on Services Computing · 5 Feb 2019 · cited 277× · DOI ↗ · more like this ·

Membership inference attacks seek to infer membership of individual training instances of a model to which an adversary has black-box access through a machine learning-as-a-service API. In providing an in-depth characterization of membership privacy risks against machine learning models, this paper presents a comprehensive study towards demystifying membership inference attacks from two complimentary perspectives. First, we provide a generalized formulation of the development of a black-box membership inference attack model. Second, we characterize the importance of model choice on model vulnerability through a systematic evaluation of a variety of machine learning models and model combinations using multiple datasets. Through formal analysis and empirical evidence from extensive experimentation, we characterize under what conditions a model may be vulnerable to such black-box…

An Anchor-Free Detection Method for Ship Targets in High-Resolution SAR Images

Zhongzhen Sun, Muchen Dai, Xiangguang Leng +4 more

Computer Vision and Pattern Recognition · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · 2021 · cited 224× · DOI ↗ · more like this ·

With the rapid development of earth observation technology, high-resolution synthetic aperture radar (HR SAR) imaging satellites could provide more observational information for maritime surveillance. However, there are still some problems to detect ship targets in HR SAR images due to the complex surroundings, targets defocusing, and diversity of the scales. In this article, an anchor-free method is proposed for ship target detection in HR SAR images. First, fully convolutional one-stage object detection (FCOS) as the base network is applied to detect ship targets, achieving better detection performance through pixel-by-pixel prediction of the image. Second, the category-position (CP) module is proposed to optimize the position regression branch features in the FCOS network. This module can improve target positioning performance in complex scenes by generating guidance vector from the…

Learning and Evaluating General Linguistic Intelligence

Dani Yogatama, Cyprien de Masson d'Autume, Jerome Connor +8 more

cs.LG · 31 Jan 2019 · cited 220× · arXiv ↗ · more like this ·

We define general linguistic intelligence as the ability to reuse previously acquired knowledge about a language's lexicon, syntax, semantics, and pragmatic conventions to adapt to new tasks quickly. Using this definition, we analyze state-of-the-art natural language understanding models and conduct an extensive empirical investigation to evaluate them against these criteria through a series of experiments that assess the task-independence of the knowledge being acquired by the learning process. In addition to task performance, we propose a new evaluation metric based on an online encoding of the test data that quantifies how quickly an existing agent (model) learns a new task. Our results show that while the field has made impressive progress in terms of model architectures that generalize to many tasks, these models still require a lot of in-domain training examples (e.g., for fine…

Perceived Discrimination and Cognition in Older African Americans

Lisa L. Barnes, T. T. Lewis, Christopher T. Begeny +3 more

Sociology and Political Science · Journal of the International Neuropsychological Society · 18 May 2012 · cited 192× · DOI ↗ · more like this ·

Existing evidence suggests that psychosocial stress is associated with cognitive impairment in older adults. Perceived discrimination is a persistent stressor in African Americans that has been associated with several adverse mental and physical health outcomes. To our knowledge, the association of discrimination with cognition in older African Americans has not been examined. In a cohort of 407 older African Americans without dementia (mean age = 72.9; SD = 6.4), we found that a higher level of perceived discrimination was related to poorer cognitive test performance, particularly episodic memory (estimate = -0.03; SE = .013; p < .05) and perceptual speed tests (estimate = -0.04; SE = .015; p < .05). The associations were unchanged after adjusting for demographics and vascular risk factors, but were attenuated after adjustment for depressive symptoms (Episodic memory estimate = -0.02…

A Mutual Information Maximization Perspective of Language Representation Learning

Lingpeng Kong, Cyprien de Masson d'Autume, Wang Ling +3 more

cs.CL · October 2019 · cited 181× · arXiv ↗ · more like this ·

We show state-of-the-art word representation learning methods maximize an objective function that is a lower bound on the mutual information between different parts of a word sequence (i.e., a sentence). Our formulation provides an alternative perspective that unifies classical word embedding models (e.g., Skip-gram) and modern contextual embeddings (e.g., BERT, XLNet). In addition to enhancing our theoretical understanding of these methods, our derivation leads to a principled framework that can be used to construct new self-supervised tasks. We provide an example by drawing inspirations from related methods based on mutual information maximization that have been successful in computer vision, and introduce a simple self-supervised objective that maximizes the mutual information between a global sentence representation and n-grams in the sentence. Our analysis offers a holistic view of…

Dynamic Coarse-to-Fine Learning for Oriented Tiny Object Detection

Chang Xu, Jian Ding, Jinwang Wang +4 more

cs.CV · 18 Apr 2023 · cited 180× · arXiv ↗ · more like this ·

Detecting arbitrarily oriented tiny objects poses intense challenges to existing detectors, especially for label assignment. Despite the exploration of adaptive label assignment in recent oriented object detectors, the extreme geometry shape and limited feature of oriented tiny objects still induce severe mismatch and imbalance issues. Specifically, the position prior, positive sample feature, and instance are mismatched, and the learning of extreme-shaped objects is biased and unbalanced due to little proper feature supervision. To tackle these issues, we propose a dynamic prior along with the coarse-to-fine assigner, dubbed DCFL. For one thing, we model the prior, label assignment, and object representation all in a dynamic manner to alleviate the mismatch issue. For another, we leverage the coarse prior matching and finer posterior constraint to dynamically assign labels, providing…

Social Collaborative Filtering by Trust

Bo Yang, Lei Yu, Jiming Liu +1 more

Information Systems · IEEE Transactions on Pattern Analysis and Machine Intelligence · September 2016 · cited 164× · DOI ↗ · more like this ·

Recommender systems are used to accurately and actively provide users with potentially interesting information or services. Collaborative filtering is a widely adopted approach to recommendation, but sparse data and cold-start users are often barriers to providing high quality recommendations. To address such issues, we propose a novel method that works to improve the performance of collaborative filtering recommendations by integrating sparse rating data given by users and sparse social trust network among these same users. This is a model-based method that adopts matrix factorization technique that maps users into low-dimensional latent feature spaces in terms of their trust relationship, and aims to more accurately reflect the users reciprocal influence on the formation of their own opinions and to learn better preferential patterns of users for high-quality recommendations. We use…

Stochastic Load Balancing for Virtual Resource Management in Datacenters

Lei Yu, Liuhua Chen, Zhipeng Cai +3 more

Information Systems · IEEE Transactions on Cloud Computing · 4 Feb 2016 · cited 150× · DOI ↗ · more like this ·

Cloud computing offers a cost-effective and elastic computing paradigm that facilitates large scale data storage and analytics. By deploying virtualization technologies in the datacenter, cloud enables efficient resource management and isolation for various big data applications. Since the hotspots (i.e., overloaded machines) can degrade the performance of these applications, virtual machine migration has been utilized to perform load balancing in the datacenters to eliminate hotspots and guarantee Service Level Agreements (SLAs). However, the previous load balancing schemes make migration decisions based on deterministic resource demand estimation and workload characterization, without considering their stochastic properties. By studying real world traces, we show that the resource demand and workload of virtual machines are highly dynamic and bursty, which can cause these schemes to…

Consensus group stable feature selection

Steven Loscalzo, Lei Yu, Chris Ding

Computer Vision and Pattern Recognition · 28 Jun 2009 · cited 141× · DOI ↗ · more like this ·

Stability is an important yet under-addressed issue in feature selection from high-dimensional and small sample data. In this paper, we show that stability of feature selection has a strong dependency on sample size. We propose a novel framework for stable feature selection which first identifies consensus feature groups from subsampling of training samples, and then performs feature selection by treating each consensus feature group as a single entity. Experiments on both synthetic and real-world data sets show that an algorithm developed under this framework is effective at alleviating the problem of small sample size and leads to more stable feature selection results and comparable or better generalization performance than state-of-the-art feature selection algorithms. Synthetic data sets and algorithm source code are available at http://www.cs.binghamton.edu/~lyu/KDD09/.

Deep Constrained Least Squares for Blind Image Super-Resolution

Ziwei Luo, Haibin Huang, Lei Yu +3 more

eess.IV · February 2022 · cited 137× · arXiv ↗ · more like this ·

In this paper, we tackle the problem of blind image super-resolution(SR) with a reformulated degradation model and two novel modules. Following the common practices of blind SR, our method proposes to improve both the kernel estimation as well as the kernel-based high-resolution image restoration. To be more specific, we first reformulate the degradation model such that the deblurring kernel estimation can be transferred into the low-resolution space. On top of this, we introduce a dynamic deep linear filter module. Instead of learning a fixed kernel for all images, it can adaptively generate deblurring kernel weights conditional on the input and yield a more robust kernel estimation. Subsequently, a deep constrained least square filtering module is applied to generate clean features based on the reformulation and estimated kernel. The deblurred feature and the low input image feature…

Fast Correlation Based Filter (FCBF) with a different search strategy

Baris Senliol, Gökhan Gülgezen, Lei Yu +1 more

Artificial Intelligence · Istanbul Technical University Academic Open Archive (Istanbul Technical University) · October 2008 · cited 135× · DOI ↗ · more like this ·

In this paper we describe an extension of the information theoretical FCBF (Fast Correlation Based Feature Selection) algorithm. The extension, called FCBF#, enables FCBF to select any given size of feature subset and it selects features in a different order than the FCBF. We find out that the extended FCBF algorithm results in more accurate classifiers.

LLaMA-Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efficient Fine-Tuning

Junyi Lu, Lei Yu, Xiaojia Li +2 more

cs.SE · August 2023 · cited 132× · arXiv ↗ · more like this ·

The automation of code review activities, a long-standing pursuit in software engineering, has been primarily addressed by numerous domain-specific pre-trained models. Despite their success, these models frequently demand extensive resources for pre-training from scratch. In contrast, Large Language Models (LLMs) provide an intriguing alternative, given their remarkable capabilities when supplemented with domain-specific knowledge. However, their potential for automating code review tasks remains largely unexplored. In response to this research gap, we present LLaMA-Reviewer, an innovative framework that leverages the capabilities of LLaMA, a popular LLM, in the realm of code review. Mindful of resource constraints, this framework employs parameter-efficient fine-tuning (PEFT) methods, delivering high performance while using less than 1% of trainable parameters. An extensive evaluation…

Unsupervised Recurrent Neural Network Grammars

Yoon Kim, Alexander M. Rush, Lei Yu +3 more

cs.CL · April 2019 · cited 123× · arXiv ↗ · more like this ·

Recurrent neural network grammars (RNNG) are generative models of language which jointly model syntax and surface structure by incrementally generating a syntax tree and sentence in a top-down, left-to-right order. Supervised RNNGs achieve strong language modeling and parsing performance, but require an annotated corpus of parse trees. In this work, we experiment with unsupervised learning of RNNGs. Since directly marginalizing over the space of latent trees is intractable, we instead apply amortized variational inference. To maximize the evidence lower bound, we develop an inference network parameterized as a neural CRF constituency parser. On language modeling, unsupervised RNNGs perform as well their supervised counterparts on benchmarks in English and Chinese. On constituency grammar induction, they are competitive with recent neural language models that induce tree structures from…

Towards Demystifying Membership Inference Attacks

Stacey Truex, Ling Liu, Mehmet Emre Gursoy +2 more

cs.CR · July 2018 · cited 120× · arXiv ↗ · more like this ·

China's strategic partnership with Latin America: a fulcrum in China's rise

Lei Yu

Political Science and International Relations · International Affairs · September 2015 · cited 119× · DOI ↗ · more like this ·

Journal Article China's strategic partnership with Latin America: a fulcrum in China's rise Get access LEI YU LEI YU 1Assistant Professor at the School of International Relations, Renmin University, Beijing and Guest Associate Professor at Beijing University of Civil Engineering and Architecture Search for other works by this author on: Oxford Academic Google Scholar International Affairs, Volume 91, Issue 5, September 2015, Pages 1047–1068, https://doi.org/10.1111/1468-2346.12397 Published: 16 September 2015

Mild Cognitive Impairment and Susceptibility to Scams in Old Age

S. Duke Han, Patricia A. Boyle, Bryan D. James +2 more

Demography · Journal of Alzheimer s Disease · 14 Dec 2015 · cited 117× · DOI ↗ · more like this ·

BACKGROUND: Falling victim to financial scams can have a significant impact upon social and financial wellbeing and independence. A large proportion of scam victims are older adults, but whether older victims with mild cognitive impairment (MCI) are at higher risk remains unknown. OBJECTIVE: We tested the hypothesis that older persons with MCI exhibit greater susceptibility to scams compared to those without cognitive impairment. METHODS: Seven hundred and thirty older adults without dementia were recruited from the Rush Memory and Aging Project, a community-based epidemiologic study of aging. Participants completed a five-item self-report measure of susceptibility to scams, a battery of cognitive measures, and clinical diagnostic evaluations. RESULTS: In models adjusted for age, education, and gender, the presence of MCI was associated with greater susceptibility to scams (B = 0.125…