Author

Hongming Zhang

234 papers · 1996–2026

1996peak 37 in 20232026

Frequent co-authors

Papers

showing 1–25 of 234 RSS

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448 more

cs.CL · June 2022 · cited 2,670× · arXiv ↗ · more like this ·

Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabilities are as yet poorly characterized. In order to inform future research, prepare for disruptive new model capabilities, and ameliorate socially harmful effects, it is vital that we understand the present and near-future capabilities and limitations of language models. To address this challenge, we introduce the Beyond the Imitation Game benchmark (BIG-bench). BIG-bench currently consists of 204 tasks, contributed by 450 authors across 132 institutions. Task topics are diverse, drawing problems from linguistics, childhood development, math, common-sense reasoning, biology, physics, social bias, software development, and beyond. BIG-bench focuses on tasks that are believed to be beyond the capabilities of…

Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition

Wenchao Zhang, Shiguang Shan, Wen Gao +2 more

Computer Vision and Pattern Recognition · 2005 · cited 1,025× · DOI ↗ · more like this ·

For years, researchers in face recognition area have been representing and recognizing faces based on subspace discriminant analysis or statistical learning. Nevertheless, these approaches are always suffering from the generalizability problem. This paper proposes a novel non-statistics based face representation approach, local Gabor binary pattern histogram sequence (LGBPHS), in which training procedure is unnecessary to construct the face model, so that the generalizability problem is naturally avoided. In this approach, a face image is modeled as a "histogram sequence" by concatenating the histograms of all the local regions of all the local Gabor magnitude binary pattern maps. For recognition, histogram intersection is used to measure the similarity of different LGBPHSs and the nearest neighborhood is exploited for final classification. Additionally, we have further proposed to…

WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models

Hongliang He, Wenlin Yao, Kaixin Ma +5 more

cs.CL · January 2024 · cited 472× · arXiv ↗ · more like this ·

The rapid advancement of large language models (LLMs) has led to a new era marked by the development of autonomous applications in real-world scenarios, which drives innovation in creating advanced web agents. Existing web agents typically only handle one input modality and are evaluated only in simplified web simulators or static web snapshots, greatly limiting their applicability in real-world scenarios. To bridge this gap, we introduce WebVoyager, an innovative Large Multimodal Model (LMM) powered web agent that can complete user instructions end-to-end by interacting with real-world websites. Moreover, we establish a new benchmark by compiling real-world tasks from 15 popular websites and introduce an automatic evaluation protocol leveraging multimodal understanding abilities of GPT-4V to evaluate open-ended web agents. We show that WebVoyager achieves a 59.1% task success rate on…

Multilingual and Multi-Aspect Hate Speech Analysis

Nedjma Ousidhoum, Zizheng Lin, Hongming Zhang +2 more

cs.CL · 29 Aug 2019 · cited 363× · arXiv ↗ · more like this ·

Current research on hate speech analysis is typically oriented towards monolingual and single classification tasks. In this paper, we present a new multilingual multi-aspect hate speech analysis dataset and use it to test the current state-of-the-art multilingual multitask learning approaches. We evaluate our dataset in various classification settings, then we discuss how to leverage our annotations in order to improve hate speech detection and classification in general.

A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation

Neeraj Varshney, Wenlin Yao, Hongming Zhang +2 more

cs.CL · July 2023 · cited 277× · arXiv ↗ · more like this ·

Recently developed large language models have achieved remarkable success in generating fluent and coherent text. However, these models often tend to 'hallucinate' which critically hampers their reliability. In this work, we address this crucial problem and propose an approach that actively detects and mitigates hallucinations during the generation process. Specifically, we first identify the candidates of potential hallucination leveraging the model's logit output values, check their correctness through a validation procedure, mitigate the detected hallucinations, and then continue with the generation process. Through extensive experiments with GPT-3.5 (text-davinci-003) on the 'article generation task', we first demonstrate the individual efficacy of our detection and mitigation techniques. Specifically, the detection technique achieves a recall of ~88% and the mitigation technique…

Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models

Wenhao Yu, Hongming Zhang, Xiaoman Pan +3 more

cs.CL · November 2023 · cited 248× · arXiv ↗ · more like this ·

Retrieval-augmented language models (RALMs) represent a substantial advancement in the capabilities of large language models, notably in reducing factual hallucination by leveraging external knowledge sources. However, the reliability of the retrieved information is not always guaranteed. The retrieval of irrelevant data can lead to misguided responses, and potentially causing the model to overlook its inherent knowledge, even when it possesses adequate information to address the query. Moreover, standard RALMs often struggle to assess whether they possess adequate knowledge, both intrinsic and retrieved, to provide an accurate answer. In situations where knowledge is lacking, these systems should ideally respond with "unknown" when the answer is unattainable. In response to these challenges, we introduces Chain-of-Noting (CoN), a novel approach aimed at improving the robustness of…

Rethinking with Retrieval: Faithful Large Language Model Inference

Hangfeng He, Hongming Zhang, Dan Roth

cs.CL · 31 Dec 2022 · cited 213× · arXiv ↗ · more like this ·

Despite the success of large language models (LLMs) in various natural language processing (NLP) tasks, the stored knowledge in these models may inevitably be incomplete, out-of-date, or incorrect. This motivates the need to utilize external knowledge to assist LLMs. Unfortunately, current methods for incorporating external knowledge often require additional training or fine-tuning, which can be costly and may not be feasible for LLMs. To address this issue, we propose a novel post-processing approach, rethinking with retrieval (RR), which retrieves relevant external knowledge based on the decomposed reasoning steps obtained from the chain-of-thought (CoT) prompting. This lightweight approach does not require additional training or fine-tuning and is not limited by the input length of LLMs. We evaluate the effectiveness of RR through extensive experiments with GPT-3 on three complex…

Scalable Multiplex Network Embedding

Hongming Zhang, Liwei Qiu, Lingling Yi +1 more

Artificial Intelligence · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · July 2018 · cited 209× · DOI ↗ · more like this ·

Network embedding has been proven to be helpful for many real-world problems. In this paper, we present a scalable multiplex network embedding model to represent information of multi-type relations into a unified embedding space. To combine information of different types of relations while maintaining their distinctive properties, for each node, we propose one high-dimensional common embedding and a lower-dimensional additional embedding for each type of relation. Then multiple relations can be learned jointly based on a unified network embedding model. We conduct experiments on two tasks: link prediction and node classification using six different multiplex networks. On both tasks, our model achieved better or comparable performance compared to current state-of-the-art models with less memory use.

ASER: A Large-scale Eventuality Knowledge Graph

Hongming Zhang, Xin Liu, Haojie Pan +2 more

cs.AI · May 2019 · cited 187× · arXiv ↗ · more like this ·

Understanding human's language requires complex world knowledge. However, existing large-scale knowledge graphs mainly focus on knowledge about entities while ignoring knowledge about activities, states, or events, which are used to describe how entities or things act in the real world. To fill this gap, we develop ASER (activities, states, events, and their relations), a large-scale eventuality knowledge graph extracted from more than 11-billion-token unstructured textual data. ASER contains 15 relation types belonging to five categories, 194-million unique eventualities, and 64-million unique edges among them. Both intrinsic and extrinsic evaluations demonstrate the quality and effectiveness of ASER.

Joint Constrained Learning for Event-Event Relation Extraction

Haoyu Wang, Muhao Chen, Hongming Zhang +1 more

cs.CL · October 2020 · cited 134× · arXiv ↗ · more like this ·

Understanding natural language involves recognizing how multiple event mentions structurally and temporally interact with each other. In this process, one can induce event complexes that organize multi-granular events with temporal order and membership relations interweaving among them. Due to the lack of jointly labeled data for these relational phenomena and the restriction on the structures they articulate, we propose a joint constrained learning framework for modeling event-event relations. Specifically, the framework enforces logical constraints within and across multiple temporal and subevent relations by converting these constraints into differentiable learning objectives. We show that our joint constrained learning approach effectively compensates for the lack of jointly labeled data, and outperforms SOTA methods on benchmarks for both temporal relation extraction and event…

TransOMCS: From Linguistic Graphs to Commonsense Knowledge

Hongming Zhang, Daniel Khashabi, Yangqiu Song +1 more

cs.AI · May 2020 · cited 101× · arXiv ↗ · more like this ·

Commonsense knowledge acquisition is a key problem for artificial intelligence. Conventional methods of acquiring commonsense knowledge generally require laborious and costly human annotations, which are not feasible on a large scale. In this paper, we explore a practical way of mining commonsense knowledge from linguistic graphs, with the goal of transferring cheap knowledge obtained with linguistic patterns into expensive commonsense knowledge. The result is a conversion of ASER [Zhang et al., 2020], a large-scale selectional preference knowledge resource, into TransOMCS, of the same representation as ConceptNet [Liu and Singh, 2004] but two orders of magnitude larger. Experimental results demonstrate the transferability of linguistic knowledge to commonsense knowledge and the effectiveness of the proposed approach in terms of quantity, novelty, and quality. TransOMCS is publicly…

AMCRN: Few-Shot Learning for Automatic Modulation Classification

Quan Zhou, Ronghui Zhang, Junsheng Mu +3 more

Artificial Intelligence · IEEE Communications Letters · 14 Dec 2021 · cited 71× · DOI ↗ · more like this ·

Deep learning (DL) has been widely applied in automatic modulation classification (AMC), while the superb performance highly depends on high-quality datasets. Motivated by this, the AMC under few-shot conditions is considered in this letter, where a novel network architecture is proposed, namely automatic modulation classification relation network (AMCRN), and verified with the baseline methods. Experimental results state that the accuracy of proposed AMCRN exceeds 90% and 10% to 50% improvements are obtained compared with classical schemes when the signal-to-noise ratio (SNR) is greater than −2 dB.

Zero-shot Event Extraction via Transfer Learning: Challenges and Insights

Qing Lyu, Hongming Zhang, Elior Sulem +1 more

Artificial Intelligence · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · 2021 · cited 68× · DOI ↗ · more like this ·

Qing Lyu, Hongming Zhang, Elior Sulem, Dan Roth. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). 2021.

Unsupervised Label-aware Event Trigger and Argument Classification

Hongming Zhang, Haoyu Wang, Dan Roth

cs.CL · December 2020 · cited 63× · arXiv ↗ · more like this ·

Identifying events and mapping them to pre-defined event types has long been an important natural language processing problem. Most previous work has been heavily relying on labor-intensive and domain-specific annotations while ignoring the semantic meaning contained in the labels of the event types. As a result, the learned models cannot effectively generalize to new domains, where new event types could be introduced. In this paper, we propose an unsupervised event extraction pipeline, which first identifies events with available tools (e.g., SRL) and then automatically maps them to pre-defined event types with our proposed unsupervised classification model. Rather than relying on annotated data, our model matches the semantics of identified events with those of event type labels. Specifically, we leverage pre-trained language models to contextually represent pre-defined types for both…

WinoWhy: A Deep Diagnosis of Essential Commonsense Knowledge for Answering Winograd Schema Challenge

Hongming Zhang, Xinran Zhao, Yangqiu Song

cs.AI · 12 May 2020 · cited 61× · arXiv ↗ · more like this ·

In this paper, we present the first comprehensive categorization of essential commonsense knowledge for answering the Winograd Schema Challenge (WSC). For each of the questions, we invite annotators to first provide reasons for making correct decisions and then categorize them into six major knowledge categories. By doing so, we better understand the limitation of existing methods (i.e., what kind of knowledge cannot be effectively represented or inferred with existing methods) and shed some light on the commonsense knowledge that we need to acquire in the future for better commonsense reasoning. Moreover, to investigate whether current WSC models can understand the commonsense or they simply solve the WSC questions based on the statistical bias of the dataset, we leverage the collected reasons to develop a new task called WinoWhy, which requires models to distinguish plausible reasons…

Providing throughput differentiation for TCP flows using adaptive two-color marking and two-level AQM

Yossi Chait, C.V. Hollot, Vishal Misra +3 more

Computer Networks and Communications · Scholarworks (University of Massachusetts Amherst) · 25 Jun 2003 · cited 53× · DOI ↗ · more like this ·

In this paper we propose a new paradigm for a Differentiated Service (DiffServ) network consisting of two-color marking at the edges of the network using token buckets coupled with differential treatment in the core. Using fluid-flow modelling, we present existence conditions for token-bucket rates and differential marking probabilities at the core that result in all edges receiving at least their minimum guaranteed rates. We then present an integrated DiffServ architecture comprising of an active rate management controller at the marking edge and a two-level active queue management controller at the core. The validity of the fluid flow model and performance of this new scheme are verified using ns simulations.

Query2Particles: Knowledge Graph Reasoning with Particle Embeddings

Jiaxin Bai, Zihao Wang, Hongming Zhang +1 more

cs.CL · 27 Apr 2022 · cited 52× · arXiv ↗ · more like this ·

Answering complex logical queries on incomplete knowledge graphs (KGs) with missing edges is a fundamental and important task for knowledge graph reasoning. The query embedding method is proposed to answer these queries by jointly encoding queries and entities to the same embedding space. Then the answer entities are selected according to the similarities between the entity embeddings and the query embedding. As the answers to a complex query are obtained from a combination of logical operations over sub-queries, the embeddings of the answer entities may not always follow a uni-modal distribution in the embedding space. Thus, it is challenging to simultaneously retrieve a set of diverse answers from the embedding space using a single and concentrated query representation such as a vector or a hyper-rectangle. To better cope with queries with diversified answers, we propose…

ASER: Towards Large-scale Commonsense Knowledge Acquisition via Higher-order Selectional Preference over Eventualities

Hongming Zhang, Xin Liu, Haojie Pan +4 more

cs.AI · April 2021 · cited 51× · arXiv ↗ · more like this ·

Commonsense knowledge acquisition and reasoning have long been a core artificial intelligence problem. However, in the past, there has been a lack of scalable methods to collect commonsense knowledge. In this paper, we propose to develop principles for collecting commonsense knowledge based on selectional preference. We generalize the definition of selectional preference from one-hop linguistic syntactic relations to higher-order relations over linguistic graphs. Unlike previous commonsense knowledge definition (e.g., ConceptNet), selectional preference (SP) knowledge only relies on statistical distribution over linguistic graphs, which can be efficiently and accurately acquired from the unlabeled corpus with modern tools. Following this principle, we develop a large-scale eventuality (a linguistic term covering activity, state, and event)-based knowledge graph ASER, where each…

Analogous Process Structure Induction for Sub-event Sequence Prediction

Hongming Zhang, Muhao Chen, Haoyu Wang +2 more

cs.AI · 16 Oct 2020 · cited 49× · arXiv ↗ · more like this ·

Computational and cognitive studies of event understanding suggest that identifying, comprehending, and predicting events depend on having structured representations of a sequence of events and on conceptualizing (abstracting) its components into (soft) event categories. Thus, knowledge about a known process such as "buying a car" can be used in the context of a new but analogous process such as "buying a house". Nevertheless, most event understanding work in NLP is still at the ground level and does not consider abstraction. In this paper, we propose an Analogous Process Structure Induction APSI framework, which leverages analogies among processes and conceptualization of sub-event instances to predict the whole sub-event sequence of previously unseen open-domain processes. As our experiments and analysis indicate, APSI supports the generation of meaningful sub-event sequences for…

Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schema

Yanai Elazar, Hongming Zhang, Yoav Goldberg +1 more

cs.CL · April 2021 · cited 48× · arXiv ↗ · more like this ·

The Winograd Schema (WS) has been proposed as a test for measuring commonsense capabilities of models. Recently, pre-trained language model-based approaches have boosted performance on some WS benchmarks but the source of improvement is still not clear. This paper suggests that the apparent progress on WS may not necessarily reflect progress in commonsense reasoning. To support this claim, we first show that the current evaluation method of WS is sub-optimal and propose a modification that uses twin sentences for evaluation. We also propose two new baselines that indicate the existence of artifacts in WS benchmarks. We then develop a method for evaluating WS-like sentences in a zero-shot setting to account for the commonsense reasoning abilities acquired during the pretraining and observe that popular language models perform randomly in this setting when using our more strict…

RESIN: A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System

Haoyang Wen, Ying Lin, Tuan Lai +9 more

Artificial Intelligence · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · 2021 · cited 45× · DOI ↗ · more like this ·

Haoyang Wen, Ying Lin, Tuan Lai, Xiaoman Pan, Sha Li, Xudong Lin, Ben Zhou, Manling Li, Haoyu Wang, Hongming Zhang, Xiaodong Yu, Alexander Dong, Zhenhailong Wang, Yi Fung, Piyush Mishra, Qing Lyu, Dídac Surís, Brian Chen, Susan Windisch Brown, Martha Palmer, Chris Callison-Burch, Carl Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang, Heng Ji. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Demonstrations. 2021.

Salience Allocation as Guidance for Abstractive Summarization

Fei Wang, Kaiqiang Song, Hongming Zhang +6 more

cs.CL · 22 Oct 2022 · cited 43× · arXiv ↗ · more like this ·

Abstractive summarization models typically learn to capture the salient information from scratch implicitly. Recent literature adds extractive summaries as guidance for abstractive summarization models to provide hints of salient content and achieves better performance. However, extractive summaries as guidance could be over strict, leading to information loss or noisy signals. Furthermore, it cannot easily adapt to documents with various abstractiveness. As the number and allocation of salience content pieces vary, it is hard to find a fixed threshold deciding which content should be included in the guidance. In this paper, we propose a novel summarization approach with a flexible and reliable salience guidance, namely SEASON (SaliencE Allocation as Guidance for Abstractive SummarizatiON). SEASON utilizes the allocation of salience expectation to guide abstractive summarization and…

METGEN: A Module-Based Entailment Tree Generation Framework for Answer Explanation

Ruixin Hong, Hongming Zhang, Xintong Yu +1 more

cs.CL · 5 May 2022 · cited 42× · arXiv ↗ · more like this ·

Knowing the reasoning chains from knowledge to the predicted answers can help construct an explainable question answering (QA) system. Advances on QA explanation propose to explain the answers with entailment trees composed of multiple entailment steps. While current work proposes to generate entailment trees with end-to-end generative models, the steps in the generated trees are not constrained and could be unreliable. In this paper, we propose METGEN, a Module-based Entailment Tree GENeration framework that has multiple modules and a reasoning controller. Given a question and several supporting knowledge, METGEN can iteratively generate the entailment tree by conducting single-step entailment with separate modules and selecting the reasoning flow with the controller. As each module is guided to perform a specific type of entailment reasoning, the steps generated by METGEN are more…

Zero-shot Label-Aware Event Trigger and Argument Classification

Hongming Zhang, Haoyu Wang, Dan Roth

Artificial Intelligence · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · 2021 · cited 36× · DOI ↗ · more like this ·

Identifying events and mapping them to a pre-defined taxonomy of event types has long been an important NLP problem. Most previous work has relied heavily on labor-intensive, domain-specific, annotation, ignoring the semantic meaning of the event types' labels. Consequently, the learned models cannot effectively generalize to new label taxonomies and domains. We propose a zero-shot event extraction approach, which first identifies events with existing tools (e.g., SRL) and then maps them to a given taxonomy of event types in a zero-shot manner. Specifically, we leverage label representations induced by pre-trained language models, and map identified events to the target types via representation similarity. To semantically type the events' arguments, we further use the definition of the events (e.g., argument of type "Victim" appears as the argument of event of type "Attack") as global…

Joint Multicast Beamforming and Relay Design for Maritime Communication Systems

Ruiyang Duan, Jingjing Wang, Hongming Zhang +2 more

Computer Networks and Communications · IEEE Transactions on Green Communications and Networking · 15 Oct 2019 · cited 36× · DOI ↗ · more like this ·

With growing human maritime activities, supporting low-cost and high-speed information services for users at sea has become imperative. However, traditional means of maritime communications fail to provide high rate services due to their high cost and limited bandwidth. In this paper, considering a base station ashore and several offshore relay nodes, we propose a cooperative multicast communication scheme for maritime users relying on joint beamforming (BF) optimization and relay design. Specifically, we decompose our proposed joint optimization problem into two subproblems, which can be solved by the feasible point pursuit successive convex approximation approach. Furthermore, an alternating optimization algorithm is proposed, which imposes an exponentially increasing complexity as a function of the number of BF elements and the number of relays, when aiming for finding the globally…