Author

Lei Li

3,441 papers · 1753–2026

1967peak 357 in 20242026

Frequent co-authors

H. Liang 1,442
Z. Y. Zhang 1,244
C. Li 1,195
H. H. Zhang 1,146
Y. Ding 1,114
J. Fang 1,045
Y. Zhang 1,032
X. Liu 994
B. Wang 934
M. Wang 933
X. Y. Zhang 930
S. X. Du 915
Y. Liu 911
W. Wang 903
K. Liu 879
Q. Liu 866

Papers

showing 1–25 of 3,441 RSS

Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis

Chaoyou Fu, Yuhan Dai, Yongdong Luo +18 more

cs.CV · May 2024 · cited 1,634× · arXiv ↗ · more like this ·

In the quest for artificial general intelligence, Multi-modal Large Language Models (MLLMs) have emerged as a focal point in recent advancements. However, the predominant focus remains on developing their capabilities in static image understanding. The potential of MLLMs in processing sequential visual data is still insufficiently explored, highlighting the absence of a comprehensive, high-quality assessment of their performance. In this paper, we introduce Video-MME, the first-ever full-spectrum, Multi-Modal Evaluation benchmark of MLLMs in Video analysis. Our work distinguishes from existing benchmarks through four key features: 1) Diversity in video types, spanning 6 primary visual domains with 30 subfields to ensure broad scenario generalizability; 2) Duration in temporal dimension, encompassing both short-, medium-, and long-term videos, ranging from 11 seconds to 1 hour, for…

Sparse R-CNN: End-to-End Object Detection with Learnable Proposals

Peize Sun, Rufeng Zhang, Yi Jiang +8 more

cs.CV · November 2020 · cited 1,555× · arXiv ↗ · more like this ·

We present Sparse R-CNN, a purely sparse method for object detection in images. Existing works on object detection heavily rely on dense object candidates, such as $k$ anchor boxes pre-defined on all grids of image feature map of size $H\times W$. In our method, however, a fixed sparse set of learned object proposals, total length of $N$, are provided to object recognition head to perform classification and location. By eliminating $HWk$ (up to hundreds of thousands) hand-designed object candidates to $N$ (e.g. 100) learnable proposals, Sparse R-CNN completely avoids all efforts related to object candidates design and many-to-one label assignment. More importantly, final predictions are directly output without non-maximum suppression post-procedure. Sparse R-CNN demonstrates accuracy, run-time and training convergence performance on par with the well-established detector baselines on…

Large Language Models are not Fair Evaluators

Peiyi Wang, Lei Li, Liang Chen +7 more

cs.CL · May 2023 · cited 1,351× · arXiv ↗ · more like this ·

In this paper, we uncover a systematic bias in the evaluation paradigm of adopting large language models~(LLMs), e.g., GPT-4, as a referee to score and compare the quality of responses generated by candidate models. We find that the quality ranking of candidate responses can be easily hacked by simply altering their order of appearance in the context. This manipulation allows us to skew the evaluation result, making one model appear considerably superior to the other, e.g., Vicuna-13B could beat ChatGPT on 66 over 80 tested queries with ChatGPT as an evaluator. To address this issue, we propose a calibration framework with three simple yet effective strategies: 1) Multiple Evidence Calibration, which requires the evaluator model to generate multiple evaluation evidence before assigning ratings; 2) Balanced Position Calibration, which aggregates results across various orders to determine…

Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations

Peiyi Wang, Lei Li, Zhihong Shao +6 more

cs.AI · December 2023 · cited 1,209× · arXiv ↗ · more like this ·

In this paper, we present an innovative process-oriented math process reward model called \textbf{Math-Shepherd}, which assigns a reward score to each step of math problem solutions. The training of Math-Shepherd is achieved using automatically constructed process-wise supervision data, breaking the bottleneck of heavy reliance on manual annotation in existing work. We explore the effectiveness of Math-Shepherd in two scenarios: 1) \textit{Verification}: Math-Shepherd is utilized for reranking multiple outputs generated by Large Language Models (LLMs); 2) \textit{Reinforcement Learning}: Math-Shepherd is employed to reinforce LLMs with step-by-step Proximal Policy Optimization (PPO). With Math-Shepherd, a series of open-source LLMs demonstrates exceptional performance. For instance, the step-by-step PPO with Math-Shepherd significantly improves the accuracy of Mistral-7B…

A Survey on In-context Learning

Qingxiu Dong, Lei Li, Damai Dai +11 more

cs.CL · January 2023 · cited 1,190× · arXiv ↗ · more like this ·

With the increasing capabilities of large language models (LLMs), in-context learning (ICL) has emerged as a new paradigm for natural language processing (NLP), where LLMs make predictions based on contexts augmented with a few examples. It has been a significant trend to explore ICL to evaluate and extrapolate the ability of LLMs. In this paper, we aim to survey and summarize the progress and challenges of ICL. We first present a formal definition of ICL and clarify its correlation to related studies. Then, we organize and discuss advanced techniques, including training strategies, prompt designing strategies, and related analysis. Additionally, we explore various ICL application scenarios, such as data engineering and knowledge updating. Finally, we address the challenges of ICL and suggest potential directions for further research. We hope that our work can encourage more research on…

BRITS: Bidirectional Recurrent Imputation for Time Series

Wei Cao, Dong Wang, Jian Li +3 more

cs.LG · 27 May 2018 · cited 1,014× · arXiv ↗ · more like this ·

Time series are widely used as signals in many classification/regression tasks. It is ubiquitous that time series contains many missing values. Given multiple correlated time series data, how to fill in missing values and to predict their class labels? Existing imputation methods often impose strong assumptions of the underlying data generating process, such as linear dynamics in the state space. In this paper, we propose BRITS, a novel method based on recurrent neural networks for missing value imputation in time series data. Our proposed method directly learns the missing values in a bidirectional recurrent dynamical system, without any specific assumption. The imputed values are treated as variables of RNN graph and can be effectively updated during the backpropagation.BRITS has three advantages: (a) it can handle multiple correlated missing values in time series; (b) it generalizes…

Dense Contrastive Learning for Self-Supervised Visual Pre-Training

Xinlong Wang, Rufeng Zhang, Chunhua Shen +2 more

cs.CV · November 2020 · cited 858× · arXiv ↗ · more like this ·

To date, most existing self-supervised learning methods are designed and optimized for image classification. These pre-trained models can be sub-optimal for dense prediction tasks due to the discrepancy between image-level prediction and pixel-level prediction. To fill this gap, we aim to design an effective, dense self-supervised learning method that directly works at the level of pixels (or local features) by taking into account the correspondence between local features. We present dense contrastive learning, which implements self-supervised learning by optimizing a pairwise contrastive (dis)similarity loss at the pixel level between two views of input images. Compared to the baseline method MoCo-v2, our method introduces negligible computation overhead (only <1% slower), but demonstrates consistently superior performance when transferring to downstream dense prediction tasks…

SOLO: Segmenting Objects by Locations

Xinlong Wang, Tao Kong, Chunhua Shen +2 more

cs.CV · December 2019 · cited 855× · arXiv ↗ · more like this ·

We present a new, embarrassingly simple approach to instance segmentation in images. Compared to many other dense prediction tasks, e.g., semantic segmentation, it is the arbitrary number of instances that have made instance segmentation much more challenging. In order to predict a mask for each instance, mainstream approaches either follow the 'detect-thensegment' strategy as used by Mask R-CNN, or predict category masks first then use clustering techniques to group pixels into individual instances. We view the task of instance segmentation from a completely new perspective by introducing the notion of "instance categories", which assigns categories to each pixel within an instance according to the instance's location and size, thus nicely converting instance mask segmentation into a classification-solvable problem. Now instance segmentation is decomposed into two classification tasks.…

VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research

Xin Wang, Jiawei Wu, Junkun Chen +3 more

cs.CV · April 2019 · cited 750× · arXiv ↗ · more like this ·

We present a new large-scale multilingual video description dataset, VATEX, which contains over 41,250 videos and 825,000 captions in both English and Chinese. Among the captions, there are over 206,000 English-Chinese parallel translation pairs. Compared to the widely-used MSR-VTT dataset, VATEX is multilingual, larger, linguistically complex, and more diverse in terms of both video and natural language descriptions. We also introduce two tasks for video-and-language research based on VATEX: (1) Multilingual Video Captioning, aimed at describing a video in various languages with a compact unified captioning model, and (2) Video-guided Machine Translation, to translate a source language description into the target language using the video information as additional spatiotemporal context. Extensive experiments on the VATEX dataset show that, first, the unified multilingual model can not…

A software framework for matchmaking based on semantic web technology

Lei Li, Ian Horrocks

Information Systems · 2003 · cited 697× · DOI ↗ · more like this ·

An important objective of the Semantic Web is to make Electronic Commerce interactions more flexible and automated. To achieve this, standardization of ontologies, message content and message protocols will be necessary.In this paper we investigate how Semantic and Web Services technologies can be used to support service advertisement and discovery in e-commerce. In particular, we describe the design and implementation of a service matchmaking prototype which uses a DAML-S based ontology and a Description Logic reasoner to compare ontology based service descriptions. We also present the results of initial experiments testing the performance of this prototype implementation in a realistic agent based e-commerce scenario.

Chinese SF-36 Health Survey: translation, cultural adaptation, validation, and normalisation

Lei Li, H M Wang, Y Shen

Health · Journal of Epidemiology & Community Health · 19 Mar 2003 · cited 651× · DOI ↗ · more like this ·

STUDY OBJECTIVE: To develop a self administered Chinese (mainland) version of the Short-Form Health Survey (SF-36) for use in health related quality of life measurements in China. DESIGN: A three stage protocol was followed including translation, tests of scaling construction and scoring assumptions, validation, and normalisation. SETTING: 1000 households in 18 communities of Hangzhou. PARTICIPANTS: 1688 respondents recruited by multi-stage mixed sampling. MAIN RESULTS: The assumption of equal intervals was violated for the vitality and mental health scales. The recoded item values were used to calculate scale scores. The clustering and ordering of item means was the same as that of the source and other two Chinese versions. The items in each scale had similar standard deviations except those in the physical functioning, boduily pain, social functioning scales. The item hypothesised…

OULU-NPU: A Mobile Face Presentation Attack Database with Real-World Variations

Zinelabinde Boulkenafet, Jukka Komulainen, Lei Li +2 more

Signal Processing · University of Oulu Repository (University of Oulu) · May 2017 · cited 575× · DOI ↗ · more like this ·

The vulnerabilities of face-based biometric systems to presentation attacks have been finally recognized but yet we lack generalized software-based face presentation attack detection (PAD) methods performing robustly in practical mobile authentication scenarios. This is mainly due to the fact that the existing public face PAD datasets are beginning to cover a variety of attack scenarios and acquisition conditions but their standard evaluation protocols do not encourage researchers to assess the generalization capabilities of their methods across these variations. In this present work, we introduce a new public face PAD database, OULU-NPU, aiming at evaluating the generalization of PAD methods in more realistic mobile authentication scenarios across three covariates: unknown environmental conditions (namely illumination and background scene), acquisition devices and presentation attack…

TempCompass: Do Video LLMs Really Understand Videos?

Yuanxin Liu, Shicheng Li, Yi Liu +6 more

cs.CV · March 2024 · cited 391× · arXiv ↗ · more like this ·

Recently, there is a surge in interest surrounding video large language models (Video LLMs). However, existing benchmarks fail to provide a comprehensive feedback on the temporal perception ability of Video LLMs. On the one hand, most of them are unable to distinguish between different temporal aspects (e.g., speed, direction) and thus cannot reflect the nuanced performance on these specific aspects. On the other hand, they are limited in the diversity of task formats (e.g., only multi-choice QA), which hinders the understanding of how temporal perception performance may vary across different types of tasks. Motivated by these two problems, we propose the \textbf{TempCompass} benchmark, which introduces a diversity of temporal aspects and task formats. To collect high-quality test data, we devise two novel strategies: (1) In video collection, we construct conflicting videos that share…

Provable Robust Watermarking for AI-Generated Text

Xuandong Zhao, Prabhanjan Ananth, Lei Li +1 more

cs.CL · June 2023 · cited 389× · arXiv ↗ · more like this ·

We study the problem of watermarking large language models (LLMs) generated text -- one of the most promising approaches for addressing the safety challenges of LLM usage. In this paper, we propose a rigorous theoretical framework to quantify the effectiveness and robustness of LLM watermarks. We propose a robust and high-quality watermark method, Unigram-Watermark, by extending an existing approach with a simplified fixed grouping strategy. We prove that our watermark method enjoys guaranteed generation quality, correctness in watermark detection, and is robust against text editing and paraphrasing. Experiments on three varying LLMs and two datasets verify that our Unigram-Watermark achieves superior detection accuracy and comparable generation quality in perplexity, thus promoting the responsible use of LLMs. Code is available at https://github.com/XuandongZhao/Unigram-Watermark.

Can We Edit Factual Knowledge by In-Context Learning?

Ce Zheng, Lei Li, Qingxiu Dong +4 more

cs.CL · 22 May 2023 · cited 386× · arXiv ↗ · more like this ·

Previous studies have shown that large language models (LLMs) like GPTs store massive factual knowledge in their parameters. However, the stored knowledge could be false or out-dated. Traditional knowledge editing methods refine LLMs via fine-tuning on texts containing specific knowledge. However, with the increasing scales of LLMs, these gradient-based approaches bring large computation costs. The trend of model-as-a-service also makes it impossible to modify knowledge in black-box LMs. Inspired by in-context learning (ICL), a new paradigm based on demonstration contexts without parameter updating, we explore whether ICL can edit factual knowledge. To answer this question, we give a comprehensive empirical study of ICL strategies. Experiments show that in-context knowledge editing (IKE), without any gradient and parameter updating, achieves a competitive success rate compared to…

Mamba-UNet: UNet-Like Pure Visual Mamba for Medical Image Segmentation

Ziyang Wang, Jian-Qing Zheng, Yichi Zhang +2 more

eess.IV · February 2024 · cited 383× · arXiv ↗ · more like this ·

In recent advancements in medical image analysis, Convolutional Neural Networks (CNN) and Vision Transformers (ViT) have set significant benchmarks. While the former excels in capturing local features through its convolution operations, the latter achieves remarkable global context understanding by leveraging self-attention mechanisms. However, both architectures exhibit limitations in efficiently modeling long-range dependencies within medical images, which is a critical aspect for precise segmentation. Inspired by the Mamba architecture, known for its proficiency in handling long sequences and global contextual information with enhanced computational efficiency as a State Space Model (SSM), we propose Mamba-UNet, a novel architecture that synergizes the U-Net in medical image segmentation with Mamba's capability. Mamba-UNet adopts a pure Visual Mamba (VMamba)-based encoder-decoder…

Why people hate your app

Bin Fu, Jialiu Lin, Lei Li +3 more

Information Systems · 11 Aug 2013 · cited 374× · DOI ↗ · more like this ·

User review is a crucial component of open mobile app markets such as the Google Play Store. How do we automatically summarize millions of user reviews and make sense out of them? Unfortunately, beyond simple summaries such as histograms of user ratings, there are few analytic tools that can provide insights into user reviews. In this paper, we propose Wiscom, a system that can analyze tens of millions user ratings and comments in mobile app markets at three different levels of detail. Our system is able to (a) discover inconsistencies in reviews; (b) identify reasons why users like or dislike a given app, and provide an interactive, zoomable view of how users' reviews evolve over time; and (c) provide valuable insights into the entire app market, identifying users' major concerns and preferences of different types of apps. Results using our techniques are reported on a 32GB dataset…

Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning

Lean Wang, Lei Li, Damai Dai +5 more

cs.CL · May 2023 · cited 303× · arXiv ↗ · more like this ·

In-context learning (ICL) emerges as a promising capability of large language models (LLMs) by providing them with demonstration examples to perform diverse tasks. However, the underlying mechanism of how LLMs learn from the provided context remains under-explored. In this paper, we investigate the working mechanism of ICL through an information flow lens. Our findings reveal that label words in the demonstration examples function as anchors: (1) semantic information aggregates into label word representations during the shallow computation layers' processing; (2) the consolidated information in label words serves as a reference for LLMs' final predictions. Based on these insights, we introduce an anchor re-weighting method to improve ICL performance, a demonstration compression technique to expedite inference, and an analysis framework for diagnosing ICL errors in GPT2-XL. The promising…

Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis

Wenhao Zhu, Hongyi Liu, Qingxiu Dong +5 more

cs.CL · April 2023 · cited 303× · arXiv ↗ · more like this ·

Large language models (LLMs) have demonstrated remarkable potential in handling multilingual machine translation (MMT). In this paper, we systematically investigate the advantages and challenges of LLMs for MMT by answering two questions: 1) How well do LLMs perform in translating massive languages? 2) Which factors affect LLMs' performance in translation? We thoroughly evaluate eight popular LLMs, including ChatGPT and GPT-4. Our empirical results show that translation capabilities of LLMs are continually involving. GPT-4 has beat the strong supervised baseline NLLB in 40.91% of translation directions but still faces a large gap towards the commercial translation system like Google Translate, especially on low-resource languages. Through further analysis, we discover that LLMs exhibit new working patterns when used for MMT. First, LLM can acquire translation ability in a…

An original face anti-spoofing approach using partial convolutional neural network

Lei Li, Xiaoyi Feng, Zinelabidine Boulkenafet +3 more

Signal Processing · IEEE Conference Proceedings · December 2016 · cited 294× · DOI ↗ · more like this ·

Recently deep Convolutional Neural Networks have been successfully applied in many computer vision tasks and achieved promising results. So some works have introduced the deep learning into face anti-spoofing. However, most approaches just use the final fully-connected layer to distinguish the real and fake faces. Inspired by the idea of each convolutional kernel can be regarded as a part filter, we extract the deep partial features from the convolutional neural network (CNN) to distinguish the real and fake faces. In our prosed approach, the CNN is fine-tuned firstly on the face spoofing datasets. Then, the block principle component analysis (PCA) method is utilized to reduce the dimensionality of features that can avoid the over-fitting problem. Lastly, the support vector machine (SVM) is employed to distinguish the real the real and fake faces. The experiments evaluated on two public…

Double Graph Based Reasoning for Document-level Relation Extraction

Shuang Zeng, Runxin Xu, Baobao Chang +1 more

cs.CL · 29 Sep 2020 · cited 290× · arXiv ↗ · more like this ·

Document-level relation extraction aims to extract relations among entities within a document. Different from sentence-level relation extraction, it requires reasoning over multiple sentences across a document. In this paper, we propose Graph Aggregation-and-Inference Network (GAIN) featuring double graphs. GAIN first constructs a heterogeneous mention-level graph (hMG) to model complex interaction among different mentions across the document. It also constructs an entity-level graph (EG), based on which we propose a novel path reasoning mechanism to infer relations between entities. Experiments on the public dataset, DocRED, show GAIN achieves a significant performance improvement (2.85 on F1) over the previous state-of-the-art. Our code is available at https://github.com/DreamInvoker/GAIN .

Personalized Prompt Learning for Explainable Recommendation

Lei Li, Yongfeng Zhang, Li Chen

cs.IR · February 2022 · cited 268× · arXiv ↗ · more like this ·

Providing user-understandable explanations to justify recommendations could help users better understand the recommended items, increase the system's ease of use, and gain users' trust. A typical approach to realize it is natural language generation. However, previous works mostly adopt recurrent neural networks to meet the ends, leaving the potentially more effective pre-trained Transformer models under-explored. In fact, user and item IDs, as important identifiers in recommender systems, are inherently in different semantic space as words that pre-trained models were already trained on. Thus, how to effectively fuse IDs into such models becomes a critical issue. Inspired by recent advancement in prompt learning, we come up with two solutions: find alternative words to represent IDs (called discrete prompt learning), and directly input ID vectors to a pre-trained model (termed…

Evolutionary route planner for unmanned air vehicles

Changwen Zheng, Lei Li, Fanjiang Xu +2 more

Computer Vision and Pattern Recognition · IEEE Transactions on Robotics · August 2005 · cited 267× · DOI ↗ · more like this ·

Based on evolutionary computation, a novel real-time route planner for unmanned air vehicles is presented. In the evolutionary route planner, the individual candidates are evaluated with respect to the workspace so that the computation of the configuration space is not required. The planner incorporates domain-specific knowledge, can handle unforeseeable changes of the environment, and take into account different kinds of mission constraints such as minimum route leg length and flying altitude, maximum turning angle, and fixed approach vector to goal position. Furthermore, the novel planner can be used to plan routes both for a single vehicle and for multiple ones. With Digital Terrain Elevation Data, the resultant routes can increase the surviving probability of the vehicles using the terrain masking effect.

On the Sentence Embeddings from Pre-trained Language Models

Bohan Li, Hao Zhou, Junxian He +3 more

cs.CL · 2 Nov 2020 · cited 265× · arXiv ↗ · more like this ·

Pre-trained contextual representations like BERT have achieved great success in natural language processing. However, the sentence embeddings from the pre-trained language models without fine-tuning have been found to poorly capture semantic meaning of sentences. In this paper, we argue that the semantic information in the BERT embeddings is not fully exploited. We first reveal the theoretical connection between the masked language model pre-training objective and the semantic similarity task theoretically, and then analyze the BERT sentence embeddings empirically. We find that BERT always induces a non-smooth anisotropic semantic space of sentences, which harms its performance of semantic similarity. To address this issue, we propose to transform the anisotropic sentence embedding distribution to a smooth and isotropic Gaussian distribution through normalizing flows that are learned…

Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completion

Xiang Chen, Ningyu Zhang, Lei Li +6 more

cs.CL · May 2022 · cited 258× · arXiv ↗ · more like this ·

Multimodal Knowledge Graphs (MKGs), which organize visual-text factual knowledge, have recently been successfully applied to tasks such as information retrieval, question answering, and recommendation system. Since most MKGs are far from complete, extensive knowledge graph completion studies have been proposed focusing on the multimodal entity, relation extraction and link prediction. However, different tasks and modalities require changes to the model architecture, and not all images/objects are relevant to text input, which hinders the applicability to diverse real-world scenarios. In this paper, we propose a hybrid transformer with multi-level fusion to address those issues. Specifically, we leverage a hybrid transformer architecture with unified input-output for diverse multimodal knowledge graph completion tasks. Moreover, we propose multi-level fusion, which integrates visual and…