arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Hongyu Lin

194 papers · 2015–2026

2015: 22016: 02017: 02018: 32019: 52020: 72021: 132022: 142023: 232024: 432025: 422026: 42
2015peak 43 in 20242026

Frequent co-authors

Papers

showing 1–25 of 194 RSS

Benchmarking Large Language Models in Retrieval-Augmented Generation

Jiawei Chen, Hongyu Lin, Xianpei Han +1 more

cs.CL · · cited 692× · arXiv ↗ · more like this ·

Retrieval-Augmented Generation (RAG) is a promising approach for mitigating the hallucination of large language models (LLMs). However, existing research lacks rigorous evaluation of the impact of retrieval-augmented generation on different large language models, which make it challenging to identify the potential bottlenecks in the capabilities of RAG for different LLMs. In this paper, we systematically investigate the impact of Retrieval-Augmented Generation on large language models. We analyze the performance of different large language models in 4 fundamental abilities required for RAG, including noise robustness, negative rejection, information integration, and counterfactual robustness. To this end, we establish Retrieval-Augmented Generation Benchmark (RGB), a new corpus for RAG evaluation in both English and Chinese. RGB divides the instances within the benchmark into 4 separate…

Unified Structure Generation for Universal Information Extraction

Yaojie Lu, Qing Liu, Dai Dai +5 more

cs.CL · · cited 665× · arXiv ↗ · more like this ·

Information extraction suffers from its varying targets, heterogeneous structures, and demand-specific schemas. In this paper, we propose a unified text-to-structure generation framework, namely UIE, which can universally model different IE tasks, adaptively generate targeted structures, and collaboratively learn general IE abilities from different knowledge sources. Specifically, UIE uniformly encodes different extraction structures via a structured extraction language, adaptively generates target extractions via a schema-based prompt mechanism - structural schema instructor, and captures the common IE abilities via a large-scale pre-trained text-to-structure model. Experiments show that UIE achieved the state-of-the-art performance on 4 IE tasks, 13 datasets, and on all supervised, low-resource, and few-shot settings for a wide range of entity, relation, event and sentiment extraction…

ShortGPT: Layers in Large Language Models are More Redundant Than You Expect

Xin Men, Mingyu Xu, Qingyu Zhang +5 more

cs.CL · · cited 452× · arXiv ↗ · more like this ·

As Large Language Models (LLMs) continue to advance in performance, their size has escalated significantly, with current LLMs containing billions or even trillions of parameters. However, in this study, we discovered that many layers of LLMs exhibit high similarity, and some layers play a negligible role in network functionality. Based on this observation, we define a metric called Block Influence (BI) to gauge the significance of each layer in LLMs. We then propose a straightforward pruning approach: layer removal, in which we directly delete the redundant layers in LLMs based on their BI scores. Experiments demonstrate that our method, which we call ShortGPT, significantly outperforms previous state-of-the-art (SOTA) methods in model pruning. Moreover, ShortGPT is orthogonal to quantization-like methods, enabling further reduction in parameters and computation. The ability to achieve…

ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases

Qiaoyu Tang, Ziliang Deng, Hongyu Lin +4 more

cs.CL · · cited 430× · arXiv ↗ · more like this ·

Enabling large language models to utilize real-world tools effectively is crucial for achieving embodied intelligence. Existing approaches to tool learning have either primarily relied on extremely large language models, such as GPT-4, to attain generalized tool-use abilities in a zero-shot manner, or utilized supervised learning to train limited scopes of tools on compact models. However, it remains uncertain whether smaller language models can achieve generalized tool-use abilities without tool-specific training. To address this question, this paper introduces ToolAlpaca, a novel framework designed to automatically generate a diverse tool-use corpus and learn generalized tool-use abilities on compact language models with minimal human intervention. Specifically, ToolAlpaca first automatically creates a highly diversified tool-use corpus by building a multi-agent simulation…

Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction

Yaojie Lu, Hongyu Lin, Jin Xu +6 more

cs.CL · · cited 347× · arXiv ↗ · more like this ·

Event extraction is challenging due to the complex structure of event records and the semantic gap between text and event. Traditional methods usually extract event records by decomposing the complex structure prediction task into multiple subtasks. In this paper, we propose Text2Event, a sequence-to-structure generation paradigm that can directly extract events from the text in an end-to-end manner. Specifically, we design a sequence-to-structure network for unified event extraction, a constrained decoding algorithm for event knowledge injection during inference, and a curriculum learning algorithm for efficient model learning. Experimental results show that, by uniformly modeling all tasks in a single model and universally predicting different labels, our method can achieve competitive performance using only record-level annotations in both supervised learning and transfer learning…

Dynamic Auxiliary Task-Based Evolutionary Multitasking for Constrained Multiobjective Optimization

Kangjia Qiao, Kunjie Yu, Boyang Qu +5 more

Computational Theory and Mathematics · IEEE Transactions on Evolutionary Computation · · cited 225× · DOI ↗ · more like this ·

When solving constrained multiobjective optimization problems (CMOPs), the utilization of infeasible solutions significantly affects algorithm’s performance because they not only maintain diversity but also provide promising search directions. In light of this situation, this article proposes a new multitasking-constrained multiobjective optimization (MTCMO) framework, in which a dynamic auxiliary task is created to assist in solving a complex CMOP (the main task) via the knowledge transfer. Moreover, the constraint boundary of the auxiliary task reduces dynamically, so that it keeps a high relatedness with the main task to continuously provide supplementary evolutionary directions. Furthermore, an improved <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula> method…

Mitigating Large Language Model Hallucinations via Autonomous Knowledge Graph-based Retrofitting

Xinyan Guan, Yanjiang Liu, Hongyu Lin +4 more

cs.CL · · cited 166× · arXiv ↗ · more like this ·

Incorporating factual knowledge in knowledge graph is regarded as a promising approach for mitigating the hallucination of large language models (LLMs). Existing methods usually only use the user's input to query the knowledge graph, thus failing to address the factual hallucination generated by LLMs during its reasoning process. To address this problem, this paper proposes Knowledge Graph-based Retrofitting (KGR), a new framework that incorporates LLMs with KGs to mitigate factual hallucination during the reasoning process by retrofitting the initial draft responses of LLMs based on the factual knowledge stored in KGs. Specifically, KGR leverages LLMs to extract, select, validate, and retrofit factual statements within the model-generated responses, which enables an autonomous knowledge verifying and refining procedure without any additional manual efforts. Experiments show that KGR…

Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases

Boxi Cao, Hongyu Lin, Xianpei Han +5 more

cs.CL · · cited 159× · arXiv ↗ · more like this ·

Previous literatures show that pre-trained masked language models (MLMs) such as BERT can achieve competitive factual knowledge extraction performance on some datasets, indicating that MLMs can potentially be a reliable knowledge source. In this paper, we conduct a rigorous study to explore the underlying predicting mechanisms of MLMs over different extraction paradigms. By investigating the behaviors of MLMs, we find that previous decent performance mainly owes to the biased prompts which overfit dataset artifacts. Furthermore, incorporating illustrative cases and external contexts improve knowledge prediction mainly due to entity type guidance and golden answer leakage. Our findings shed light on the underlying predicting mechanisms of MLMs, and strongly question the previous conclusion that current MLMs can potentially serve as reliable factual knowledge bases.

Sequence-to-Nuggets: Nested Entity Mention Detection via Anchor-Region Networks

Hongyu Lin, Yaojie Lu, Xianpei Han +1 more

cs.CL · · cited 120× · arXiv ↗ · more like this ·

Sequential labeling-based NER approaches restrict each word belonging to at most one entity mention, which will face a serious problem when recognizing nested entity mentions. In this paper, we propose to resolve this problem by modeling and leveraging the head-driven phrase structures of entity mentions, i.e., although a mention can nest other mentions, they will not share the same head word. Specifically, we propose Anchor-Region Networks (ARNs), a sequence-to-nuggets architecture for nested mention detection. ARNs first identify anchor words (i.e., possible head words) of all mentions, and then recognize the mention boundaries for each anchor word by exploiting regular phrase structures. Furthermore, we also design Bag Loss, an objective function which can train ARNs in an end-to-end manner without using any anchor word annotation. Experiments show that ARNs achieve the…

Universal Information Extraction as Unified Semantic Matching

Jie Lou, Yaojie Lu, Dai Dai +5 more

cs.CL · · cited 112× · arXiv ↗ · more like this ·

The challenge of information extraction (IE) lies in the diversity of label schemas and the heterogeneity of structures. Traditional methods require task-specific model design and rely heavily on expensive supervision, making them difficult to generalize to new schemas. In this paper, we decouple IE into two basic abilities, structuring and conceptualizing, which are shared by different tasks and schemas. Based on this paradigm, we propose to universally model various IE tasks with Unified Semantic Matching (USM) framework, which introduces three unified token linking operations to model the abilities of structuring and conceptualizing. In this way, USM can jointly encode schema and input text, uniformly extract substructures in parallel, and controllably decode target structures on demand. Empirical evaluation on 4 IE tasks shows that the proposed method achieves state-of-the-art…

ChatGPT is a Knowledgeable but Inexperienced Solver: An Investigation of Commonsense Problem in Large Language Models

Ning Bian, Xianpei Han, Le Sun +5 more

cs.CL · · cited 105× · arXiv ↗ · more like this ·

Large language models (LLMs) have made significant progress in NLP. However, their ability to memorize, represent, and leverage commonsense knowledge has been a well-known pain point. In this paper, we specifically focus on ChatGPT, a widely used and easily accessible LLM, and ask the following questions: (1) Can ChatGPT effectively answer commonsense questions? (2) Is ChatGPT aware of the underlying commonsense knowledge for answering a specific question? (3) Is ChatGPT knowledgeable in commonsense? (4) Can ChatGPT effectively leverage commonsense for answering questions? We conduct a series of experiments on 11 datasets to evaluate ChatGPT's commonsense abilities, including answering commonsense questions, identifying necessary knowledge, generating knowledge descriptions, and using knowledge descriptions to answer questions again. Experimental results show that: (1) ChatGPT can…

Evolutionary Constrained Multiobjective Optimization: Scalable High-Dimensional Constraint Benchmarks and Algorithm

Kangjia Qiao, Jing Liang, Kunjie Yu +4 more

Computational Theory and Mathematics · IEEE Transactions on Evolutionary Computation · · cited 93× · DOI ↗ · more like this ·

Evolutionary constrained multiobjective optimization has received extensive attention and research in the past two decades, and a lot of benchmarks have been proposed to test the effect of the constrained multiobjective evolutionary algorithms (CMOEAs). Specially, the constraint functions are highly correlated with the objective values, which makes the features of constraints too monotonic and differ from the properties of the real-world problems. Accordingly, previous CMOEAs cannot solve real-world problems well, which generally involve decision space constraints with multi-modal/non-linear features. Therefore, we propose a new benchmark framework and design a suite of new test functions with scalable high-dimensional decision space constraints. To be specific, different high-dimensional constraint functions and mixed linkages in variables are considered to be close to realistic…

Multiobjective Differential Evolution With Speciation for Constrained Multimodal Multiobjective Optimization

Jing Liang, Hongyu Lin, Caitong Yue +3 more

Computational Theory and Mathematics · IEEE Transactions on Evolutionary Computation · · cited 82× · DOI ↗ · more like this ·

This article proposes a novel differential evolution algorithm for solving constrained multimodal multiobjective optimization problems (CMMOPs), which may have multiple feasible Pareto-optimal solutions with identical objective vectors. In CMMOPs, due to the coexistence of multimodality and constraints, it is difficult for current algorithms to perform well in both objective and decision spaces. The proposed algorithm uses the speciation mechanism to induce niches preserving more feasible Pareto-optimal solutions and adopts an improved environment selection criterion to enhance diversity. The algorithm can not only obtain feasible solutions but also retain more well-distributed feasible Pareto-optimal solutions. Moreover, a set of constrained multimodal multiobjective test functions is developed. All these test functions have multimodal characteristics and contain multiple constraints.…

Nugget Proposal Networks for Chinese Event Detection

Hongyu Lin, Yaojie Lu, Xianpei Han +1 more

cs.CL · · cited 71× · arXiv ↗ · more like this ·

Neural network based models commonly regard event detection as a word-wise classification task, which suffer from the mismatch problem between words and event triggers, especially in languages without natural word delimiters such as Chinese. In this paper, we propose Nugget Proposal Networks (NPNs), which can solve the word-trigger mismatch problem by directly proposing entire trigger nuggets centered at each character regardless of word boundaries. Specifically, NPNs perform event detection in a character-wise paradigm, where a hybrid representation for each character is first learned to capture both structural and semantic information from both characters and words. Then based on learned representations, trigger nuggets are proposed and categorized by exploiting character compositional structures of Chinese event triggers. Experiments on both ACE2005 and TAC KBP 2017 datasets show…

Few-shot Named Entity Recognition with Self-describing Networks

Jiawei Chen, Qing Liu, Hongyu Lin +2 more

cs.CL · · cited 57× · arXiv ↗ · more like this ·

Few-shot NER needs to effectively capture information from limited instances and transfer useful knowledge from external resources. In this paper, we propose a self-describing mechanism for few-shot NER, which can effectively leverage illustrative instances and precisely transfer knowledge from external resources by describing both entity types and mentions using a universal concept set. Specifically, we design Self-describing Networks (SDNet), a Seq2Seq generation model which can universally describe mentions using concepts, automatically map novel entity types to concepts, and adaptively recognize entities on-demand. We pre-train SDNet with large-scale corpus, and conduct experiments on 8 benchmarks from different domains. Experiments show that SDNet achieves competitive performances on all benchmarks and achieves the new state-of-the-art on 6 benchmarks, which demonstrates its…

Can Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal View

Boxi Cao, Hongyu Lin, Xianpei Han +2 more

cs.CL · · cited 48× · arXiv ↗ · more like this ·

Prompt-based probing has been widely used in evaluating the abilities of pretrained language models (PLMs). Unfortunately, recent studies have discovered such an evaluation may be inaccurate, inconsistent and unreliable. Furthermore, the lack of understanding its inner workings, combined with its wide applicability, has the potential to lead to unforeseen risks for evaluating and applying PLMs in real-world applications. To discover, understand and quantify the risks, this paper investigates the prompt-based probing from a causal view, highlights three critical biases which could induce biased results and conclusions, and proposes to conduct debiasing via causal intervention. This paper provides valuable insights for the design of unbiased datasets, better probing frameworks and more reliable evaluations of pretrained language models. Furthermore, our conclusions also echo that we need…

De-biasing Distantly Supervised Named Entity Recognition via Causal Intervention

Wenkai Zhang, Hongyu Lin, Xianpei Han +1 more

cs.CL · · cited 48× · arXiv ↗ · more like this ·

Distant supervision tackles the data bottleneck in NER by automatically generating training instances via dictionary matching. Unfortunately, the learning of DS-NER is severely dictionary-biased, which suffers from spurious correlations and therefore undermines the effectiveness and the robustness of the learned models. In this paper, we fundamentally explain the dictionary bias via a Structural Causal Model (SCM), categorize the bias into intra-dictionary and inter-dictionary biases, and identify their causes. Based on the SCM, we learn de-biased DS-NER via causal interventions. For intra-dictionary bias, we conduct backdoor adjustment to remove the spurious correlations introduced by the dictionary confounder. For inter-dictionary bias, we propose a causal invariance regularizer which will make DS-NER models more robust to the perturbation of dictionaries. Experiments on four datasets…

Distilling Discrimination and Generalization Knowledge for Event Detection via Delta-Representation Learning

Yaojie Lu, Hongyu Lin, Xianpei Han +1 more

Artificial Intelligence · · cited 44× · DOI ↗ · more like this ·

Event detection systems rely on discrimination knowledge to distinguish ambiguous trigger words and generalization knowledge to detect unseen/sparse trigger words. Current neural event detection approaches focus on trigger-centric representations, which work well on distilling discrimination knowledge, but poorly on learning generalization knowledge. To address this problem, this paper proposes a -learning approach to distill discrimination and generalization knowledge by effectively decoupling, incrementally learning and adaptively fusing event representation. Experiments show that our method significantly outperforms previous approaches on unseen/sparse trigger words, and achieves state-of-the-art performance on both ACE2005 and KBP2017 datasets.

Pre-training to Match for Unified Low-shot Relation Extraction

Fangchao Liu, Hongyu Lin, Xianpei Han +2 more

cs.CL · · cited 41× · arXiv ↗ · more like this ·

Low-shot relation extraction~(RE) aims to recognize novel relations with very few or even no samples, which is critical in real scenario application. Few-shot and zero-shot RE are two representative low-shot RE tasks, which seem to be with similar target but require totally different underlying abilities. In this paper, we propose Multi-Choice Matching Networks to unify low-shot relation extraction. To fill in the gap between zero-shot and few-shot RE, we propose the triplet-paraphrase meta-training, which leverages triplet paraphrase to pre-train zero-shot label matching ability and uses meta-learning paradigm to learn few-shot instance summarizing ability. Experimental results on three different low-shot RE tasks show that the proposed method outperforms strong baselines by a large margin, and achieve the best performance on few-shot RE leaderboard.

Learning In-context Learning for Named Entity Recognition

Jiawei Chen, Yaojie Lu, Hongyu Lin +7 more

cs.CL · · cited 40× · arXiv ↗ · more like this ·

Named entity recognition in real-world applications suffers from the diversity of entity types, the emergence of new entity types, and the lack of high-quality annotations. To address the above problems, this paper proposes an in-context learning-based NER approach, which can effectively inject in-context NER ability into PLMs and recognize entities of novel types on-the-fly using only a few demonstrative instances. Specifically, we model PLMs as a meta-function $\mathcal{ λ_ {\text{instruction, demonstrations, text}}. M}$, and a new entity extractor can be implicitly constructed by applying new instruction and demonstrations to PLMs, i.e., $\mathcal{ (λ. M) }$(instruction, demonstrations) $\to$ $\mathcal{F}$ where $\mathcal{F}$ will be a new entity extractor, i.e., $\mathcal{F}$: text $\to$ entities. To inject the above in-context NER ability into PLMs, we propose a meta-function…

The Life Cycle of Knowledge in Big Language Models: A Survey

Boxi Cao, Hongyu Lin, Xianpei Han +1 more

cs.CL · · cited 38× · arXiv ↗ · more like this ·

Knowledge plays a critical role in artificial intelligence. Recently, the extensive success of pre-trained language models (PLMs) has raised significant attention about how knowledge can be acquired, maintained, updated and used by language models. Despite the enormous amount of related studies, there still lacks a unified view of how knowledge circulates within language models throughout the learning, tuning, and application processes, which may prevent us from further understanding the connections between current progress or realizing existing limitations. In this survey, we revisit PLMs as knowledge-based systems by dividing the life circle of knowledge in PLMs into five critical periods, and investigating how knowledge circulates when it is built, maintained and used. To this end, we systematically review existing studies of each period of the knowledge life cycle, summarize the…

Influence of External Information on Large Language Models Mirrors Social Cognitive Patterns

Ning Bian, Hongyu Lin, Peilin Liu +5 more

cs.CL · · cited 37× · arXiv ↗ · more like this ·

Social cognitive theory explains how people learn and acquire knowledge through observing others. Recent years have witnessed the rapid development of large language models (LLMs), which suggests their potential significance as agents in the society. LLMs, as AI agents, can observe external information, which shapes their cognition and behaviors. However, the extent to which external information influences LLMs' cognition and behaviors remains unclear. This study investigates how external statements and opinions influence LLMs' thoughts and behaviors from a social cognitive perspective. Three experiments were conducted to explore the effects of external information on LLMs' memories, opinions, and social media behavioral decisions. Sociocognitive factors, including source authority, social identity, and social role, were analyzed to investigate their moderating effects. Results showed…

Multiobjective Differential Evolution for Higher-Dimensional Multimodal Multiobjective Optimization

Jing Liang, Hongyu Lin, Caitong Yue +2 more

Computational Theory and Mathematics · IEEE/CAA Journal of Automatica Sinica · · cited 33× · DOI ↗ · more like this ·

In multimodal multiobjective optimization problems (MMOPs), there are several Pareto optimal solutions corresponding to the identical objective vector. This paper proposes a new differential evolution algorithm to solve MMOPs with higher-dimensional decision variables. Due to the increase in the dimensions of decision variables in real-world MMOPs, it is difficult for current multimodal multiobjective optimization evolutionary algorithms (MMOEAs) to find multiple Pareto optimal solutions. The proposed algorithm adopts a dual-population framework and an improved environmental selection method. It utilizes a convergence archive to help the first population improve the quality of solutions. The improved environmental selection method enables the other population to search the remaining decision space and reserve more Pareto optimal solutions through the information of the first population.…

Honey or Poison? Solving the Trigger Curse in Few-shot Event Detection via Causal Intervention

Jiawei Chen, Hongyu Lin, Xianpei Han +1 more

cs.CL · · cited 31× · arXiv ↗ · more like this ·

Event detection has long been troubled by the \emph{trigger curse}: overfitting the trigger will harm the generalization ability while underfitting it will hurt the detection performance. This problem is even more severe in few-shot scenario. In this paper, we identify and solve the trigger curse problem in few-shot event detection (FSED) from a causal view. By formulating FSED with a structural causal model (SCM), we found that the trigger is a confounder of the context and the result, which makes previous FSED methods much easier to overfit triggers. To resolve this problem, we propose to intervene on the context via backdoor adjustment during training. Experiments show that our method significantly improves the FSED on ACE05, MAVEN and KBP17 datasets.

From Discourse to Narrative: Knowledge Projection for Event Relation Extraction

Jialong Tang, Hongyu Lin, Meng Liao +5 more

cs.CL · · cited 29× · arXiv ↗ · more like this ·

Current event-centric knowledge graphs highly rely on explicit connectives to mine relations between events. Unfortunately, due to the sparsity of connectives, these methods severely undermine the coverage of EventKGs. The lack of high-quality labelled corpora further exacerbates that problem. In this paper, we propose a knowledge projection paradigm for event relation extraction: projecting discourse knowledge to narratives by exploiting the commonalities between them. Specifically, we propose Multi-tier Knowledge Projection Network (MKPNet), which can leverage multi-tier discourse knowledge effectively for event relation extraction. In this way, the labelled data requirement is significantly reduced, and implicit event relations can be effectively extracted. Intrinsic experimental results show that MKPNet achieves the new state-of-the-art performance, and extrinsic experimental…