Author

Chenguang Wang

244 papers · 1999–2026

1999peak 46 in 20262026

Frequent co-authors

Papers

showing 1–25 of 244 RSS

Co-Occurrent Features in Semantic Segmentation

Hang Zhang, Han Zhang, Chenguang Wang +1 more

Computer Vision and Pattern Recognition · June 2019 · cited 321× · DOI ↗ · more like this ·

Recent work has achieved great success in utilizing global contextual information for semantic segmentation, including increasing the receptive field and aggregating pyramid feature representations. In this paper, we go beyond global context and explore the fine-grained representation using co-occurrent features by introducing Co-occurrent Feature Model, which predicts the distribution of co-occurrent features for a given target. To leverage the semantic context in the co-occurrent features, we build an Aggregated Co-occurrent Feature (ACF) Module by aggregating the probability of the co-occurrent feature with the co-occurrent context. ACF Module learns a fine-grained spatial invariant representation to capture co-occurrent context information across the scene. Our approach significantly improves the segmentation results using FCN and achieves superior performance 54.0% mIoU on Pascal…

GluonCV and GluonNLP: Deep Learning in Computer Vision and Natural Language Processing

Jian Guo, He He, Tong He +13 more

cs.LG · July 2019 · cited 220× · arXiv ↗ · more like this ·

We present GluonCV and GluonNLP, the deep learning toolkits for computer vision and natural language processing based on Apache MXNet (incubating). These toolkits provide state-of-the-art pre-trained models, training scripts, and training logs, to facilitate rapid prototyping and promote reproducible research. We also provide modular APIs with flexible building blocks to enable efficient customization. Leveraging the MXNet ecosystem, the deep learning models in GluonCV and GluonNLP can be deployed onto a variety of platforms with different programming languages. The Apache 2.0 license has been adopted by GluonCV and GluonNLP to allow for software distribution, modification, and usage.

Language Models are Open Knowledge Graphs

Chenguang Wang, Xiao Liu, Dawn Song

cs.CL · 22 Oct 2020 · cited 149× · arXiv ↗ · more like this ·

This paper shows how to construct knowledge graphs (KGs) from pre-trained language models (e.g., BERT, GPT-2/3), without human supervision. Popular KGs (e.g, Wikidata, NELL) are built in either a supervised or semi-supervised manner, requiring humans to create knowledge. Recent deep language models automatically acquire knowledge from large-scale corpora via pre-training. The stored knowledge has enabled the language models to improve downstream NLP tasks, e.g., answering questions, and writing code and articles. In this paper, we propose an unsupervised method to cast the knowledge contained within language models into KGs. We show that KGs are constructed with a single forward pass of the pre-trained language models (without fine-tuning) over the corpora. We demonstrate the quality of the constructed KGs by comparing to two KGs (Wikidata, TAC KBP) created by humans. Our KGs also…

Language Models with Transformers

Chenguang Wang, Mu Li, Alexander J. Smola

cs.CL · April 2019 · cited 142× · arXiv ↗ · more like this ·

The Transformer architecture is superior to RNN-based models in computational efficiency. Recently, GPT and BERT demonstrate the efficacy of Transformer models on various NLP tasks using pre-trained language models on large-scale corpora. Surprisingly, these Transformer architectures are suboptimal for language model itself. Neither self-attention nor the positional encoding in the Transformer is able to efficiently incorporate the word-level sequential context crucial to language modeling. In this paper, we explore effective Transformer architectures for language model, including adding additional LSTM layers to better capture the sequential context while still keeping the computation efficient. We propose Coordinate Architecture Search (CAS) to find an effective architecture through iterative refinement of the model. Experimental results on the PTB, WikiText-2, and WikiText-103 show…

Protecting Intellectual Property of Language Generation APIs with Lexical Watermark

Xuanli He, Qiongkai Xu, Lingjuan Lyu +2 more

cs.CR · 5 Dec 2021 · cited 133× · arXiv ↗ · more like this ·

Nowadays, due to the breakthrough in natural language generation (NLG), including machine translation, document summarization, image captioning, etc NLG models have been encapsulated in cloud APIs to serve over half a billion people worldwide and process over one hundred billion word generations per day. Thus, NLG APIs have already become essential profitable services in many commercial companies. Due to the substantial financial and intellectual investments, service providers adopt a pay-as-you-use policy to promote sustainable market growth. However, recent works have shown that cloud platforms suffer from financial losses imposed by model extraction attacks, which aim to imitate the functionality and utility of the victim services, thus violating the intellectual property (IP) of cloud APIs. This work targets at protecting IP of NLG APIs by identifying the attackers who have utilized…

Propensity score-integrated power prior approach for incorporating real-world evidence in single-arm clinical studies

Chenguang Wang, Heng Li, Wei-Chen Chen +4 more

Statistics and Probability · Journal of Biopharmaceutical Statistics · 3 Sep 2019 · cited 117× · DOI ↗ · more like this ·

We are now at an amazing time for medical product development in drugs, biological products and medical devices. As a result of dramatic recent advances in biomedical science, information technology and engineering, ``big data'' from health care in the real-world have become available. Although big data may not necessarily be attuned to provide the preponderance of evidence to a clinical study, high-quality real-world data can be transformed into scientific evidence for regulatory and healthcare decision-making using proven analytical methods and techniques, such as propensity score methodology and Bayesian inference. In this paper, we extend the Bayesian power prior approach for a single-arm study (the current study) to leverage external real-world data. We use propensity score methodology to pre-select a subset of real-world data containing patients that are similar to those in the…

DeepStruct: Pretraining of Language Models for Structure Prediction

Chenguang Wang, Xiao Liu, Zui Chen +3 more

cs.CL · May 2022 · cited 103× · arXiv ↗ · more like this ·

We introduce a method for improving the structural understanding abilities of language models. Unlike previous approaches that finetune the models with task-specific augmentation, we pretrain language models on a collection of task-agnostic corpora to generate structures from text. Our structure pretraining enables zero-shot transfer of the learned knowledge that models have about the structure tasks. We study the performance of this approach on 28 datasets, spanning 10 structure prediction tasks including open information extraction, joint entity and relation extraction, named entity recognition, relation classification, semantic role labeling, event extraction, coreference resolution, factual probe, intent detection, and dialogue state tracking. We further enhance the pretraining with the task-specific training sets. We show that a 10B parameter language model transfers non-trivially…

Text Classification with Heterogeneous Information Network Kernels

Chenguang Wang, Yangqiu Song, Haoran Li +2 more

Artificial Intelligence · Proceedings of the AAAI Conference on Artificial Intelligence · 2 Mar 2016 · cited 86× · DOI ↗ · more like this ·

Text classification is an important problem with many applications. Traditional approaches represent text as a bag-of-words and build classifiers based on this representation. Rather than words, entity phrases, the relations between the entities, as well as the types of the entities and relations carry much more information to represent the texts. This paper presents a novel text as network classification framework, which introduces 1) a structured and typed heterogeneous information networks (HINs) representation of texts, and 2) a meta-path based approach to link texts. We show that with the new representation and links of texts, the structured and typed information of entities and relations can be incorporated into kernels. Particularly, we develop both simple linear kernel and indefinite kernel based on meta-paths in the HIN representation of texts, where we call them HIN-kernels.…

Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models

Zhiyuan Zhang, Lingjuan Lyu, Xingjun Ma +2 more

cs.CL · 18 Oct 2022 · cited 77× · arXiv ↗ · more like this ·

Deep Neural Networks (DNNs) are known to be vulnerable to backdoor attacks. In Natural Language Processing (NLP), DNNs are often backdoored during the fine-tuning process of a large-scale Pre-trained Language Model (PLM) with poisoned samples. Although the clean weights of PLMs are readily available, existing methods have ignored this information in defending NLP models against backdoor attacks. In this work, we take the first step to exploit the pre-trained (unfine-tuned) weights to mitigate backdoors in fine-tuned language models. Specifically, we leverage the clean pre-trained weights via two complementary techniques: (1) a two-step Fine-mixing technique, which first mixes the backdoored weights (fine-tuned on poisoned data) with the pre-trained weights, then fine-tunes the mixed weights on a small subset of clean data; (2) an Embedding Purification (E-PUR) technique, which mitigates…

Practical Membership Inference Attacks Against Large-Scale Multi-Modal Models: A Pilot Study

Myeongseob Ko, Ming Jin, Chenguang Wang +1 more

cs.LG · 29 Sep 2023 · cited 62× · arXiv ↗ · more like this ·

Membership inference attacks (MIAs) aim to infer whether a data point has been used to train a machine learning model. These attacks can be employed to identify potential privacy vulnerabilities and detect unauthorized use of personal data. While MIAs have been traditionally studied for simple classification models, recent advancements in multi-modal pre-training, such as CLIP, have demonstrated remarkable zero-shot performance across a range of computer vision tasks. However, the sheer scale of data and models presents significant computational challenges for performing the attacks. This paper takes a first step towards developing practical MIAs against large-scale multi-modal models. We introduce a simple baseline strategy by thresholding the cosine similarity between text and image features of a target point and propose further enhancing the baseline by aggregating cosine similarity…

Propensity score-integrated composite likelihood approach for augmenting the control arm of a randomized controlled trial by incorporating real-world data

Wei-Chen Chen, Chenguang Wang, Heng Li +4 more

Statistics and Probability · Journal of Biopharmaceutical Statistics · 3 May 2020 · cited 59× · DOI ↗ · more like this ·

In this paper, a propensity score-integrated composite likelihood (PSCL) approach is developed for cases in which the control arm of a two-arm randomized controlled trial (RCT) (treated vs control) is augmented with patients from real-world data (RWD) containing both clinical outcomes and covariates at the patient-level. RWD patients who were treated with the same therapy as the control arm of the RCT are considered for the augmentation. The PSCL approach first estimates the propensity score for every patient as the probability of the patient being in the RCT rather than the RWD, and then stratifies all patients into strata based on the estimated propensity scores. Within each propensity score stratum, a composite likelihood function is specified and utilized to down-weight the information contributed by the RWD source. Estimates of the stratum-specific parameters are obtained by…

Bayesian analysis of heterogeneous treatment effects for patient-centered outcomes research

Nicholas C. Henderson, Thomas A. Louis, Chenguang Wang +1 more

Statistics and Probability · Health Services and Outcomes Research Methodology · 20 Sep 2016 · cited 59× · DOI ↗ · more like this ·

Evaluation of heterogeneity of treatment effect (HTE) is an essential aspect of personalized medicine and patient-centered outcomes research. Our goal in this article is to promote the use of Bayesian methods for subgroup analysis and to lower the barriers to their implementation by describing the ways in which the companion software beanz can facilitate these types of analyses. To advance this goal, we describe several key Bayesian models for investigating HTE and outline the ways in which they are well-suited to address many of the commonly cited challenges in the study of HTE. Topics highlighted include shrinkage estimation, model choice, sensitivity analysis, and posterior predictive checking. A case study is presented in which we demonstrate the use of the methods discussed.

KnowSim: A Document Similarity Measure on Structured Heterogeneous Information Networks

Chenguang Wang, Yangqiu Song, Haoran Li +2 more

Artificial Intelligence · PubMed · November 2015 · cited 57× · DOI ↗ · more like this ·

As a fundamental task, document similarity measure has broad impact to document-based classification, clustering and ranking. Traditional approaches represent documents as bag-of-words and compute document similarities using measures like cosine, Jaccard, and dice. However, entity phrases rather than single words in documents can be critical for evaluating document relatedness. Moreover, types of entities and links between entities/words are also informative. We propose a method to represent a document as a typed heterogeneous information network (HIN), where the entities and relations are annotated with types. Multiple documents can be linked by the words and entities in the HIN. Consequently, we convert the document similarity problem to a graph distance problem. Intuitively, there could be multiple paths between a pair of documents. We propose to use the meta-path defined in HIN to…

Incorporating World Knowledge to Document Clustering via Heterogeneous Information Networks

Chenguang Wang, Yangqiu Song, Ahmed El-Kishky +3 more

Artificial Intelligence · PubMed · 7 Aug 2015 · cited 52× · DOI ↗ · more like this ·

One of the key obstacles in making learning protocols realistic in applications is the need to supervise them, a costly process that often requires hiring domain experts. We consider the framework to use the world knowledge as indirect supervision. World knowledge is general-purpose knowledge, which is not designed for any specific domain. Then the key challenges are how to adapt the world knowledge to domains and how to represent it for learning. In this paper, we provide an example of using world knowledge for domain dependent document clustering. We provide three ways to specify the world knowledge to domains by resolving the ambiguity of the entities and their types, and represent the data with world knowledge as a heterogeneous information network. Then we propose a clustering algorithm that can cluster multiple types and incorporate the sub-type information as constraints. In the…

RelSim: Relation Similarity Search in Schema-Rich Heterogeneous Information Networks

Chenguang Wang, Yizhou Sun, Yanglei Song +4 more

Artificial Intelligence · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · 30 Jun 2016 · cited 50× · DOI ↗ · more like this ·

Recent studies have demonstrated the power of modeling real world data as heterogeneous information networks (HINs) consisting of multiple types of entities and relations. Unfortunately, most of such studies (e.g., similarity search) confine discussions on the networks with only a few entity and relationship types, such as DBLP. In the real world, however, the network schema can be rather complex, such as Freebase. In such HINs with rich schema, it is often too much burden to ask users to provide explicit guidance in selecting relations for similarity search. In this paper, we study the problem of relation similarity search in schema-rich HINs. Under our problem setting, users are only asked to provide some simple relation instance examples (e.g., 〈Barack Obama, John Kerry〉 and 〈George W. Bush, Condoleezza Rice〉) as a query, and we automatically detect the latent semantic relation (LSR)…

Optical Flow Sensor/INS/Magnetometer Integrated Navigation System for MAV in GPS-Denied Environment

Chong Shen, Zesen Bai, Huiliang Cao +6 more

Computer Vision and Pattern Recognition · Journal of Sensors · 2016 · cited 49× · DOI ↗ · more like this ·

The drift of inertial navigation system (INS) will lead to large navigation error when a low-cost INS is used in microaerial vehicles (MAV). To overcome the above problem, an INS/optical flow/magnetometer integrated navigation scheme is proposed for GPS-denied environment in this paper. The scheme, which is based on extended Kalman filter, combines INS and optical flow information to estimate the velocity and position of MAV. The gyro, accelerator, and magnetometer information are fused together to estimate the MAV attitude when the MAV is at static state or uniformly moving state; and the gyro only is used to estimate the MAV attitude when the MAV is accelerating or decelerating. The MAV flight data is used to verify the proposed integrated navigation scheme, and the verification results show that the proposed scheme can effectively reduce the errors of navigation parameters and…

Joint Language Semantic and Structure Embedding for Knowledge Graph Completion

Jianhao Shen, Chenguang Wang, Linyuan Gong +1 more

cs.CL · 19 Sep 2022 · cited 48× · arXiv ↗ · more like this ·

The task of completing knowledge triplets has broad downstream applications. Both structural and semantic information plays an important role in knowledge graph completion. Unlike previous approaches that rely on either the structures or semantics of the knowledge graphs, we propose to jointly embed the semantics in the natural language description of the knowledge triplets with their structure information. Our method embeds knowledge graphs for the completion task via fine-tuning pre-trained language models with respect to a probabilistic structured loss, where the forward pass of the language models captures semantics and the loss reconstructs structures. Our extensive experiments on a variety of knowledge graph benchmarks have demonstrated the state-of-the-art performance of our method. We also show that our method can significantly improve the performance in a low-resource regime…

ASP: Learn a Universal Neural Solver!

Chenguang Wang, Zhouliang Yu, Stephen McAleer +2 more

cs.LG · March 2023 · cited 47× · arXiv ↗ · more like this ·

Applying machine learning to combinatorial optimization problems has the potential to improve both efficiency and accuracy. However, existing learning-based solvers often struggle with generalization when faced with changes in problem distributions and scales. In this paper, we propose a new approach called ASP: Adaptive Staircase Policy Space Response Oracle to address these generalization issues and learn a universal neural solver. ASP consists of two components: Distributional Exploration, which enhances the solver's ability to handle unknown distributions using Policy Space Response Oracles, and Persistent Scale Adaption, which improves scalability through curriculum learning. We have tested ASP on several challenging COPs, including the traveling salesman problem, the vehicle routing problem, and the prize collecting TSP, as well as the real-world instances from TSPLib and CVRPLib.…

Zero-Shot Information Extraction as a Unified Text-to-Triple Translation

Chenguang Wang, Xiao Liu, Zui Chen +3 more

cs.CL · 23 Sep 2021 · cited 46× · arXiv ↗ · more like this ·

We cast a suite of information extraction tasks into a text-to-triple translation framework. Instead of solving each task relying on task-specific datasets and models, we formalize the task as a translation between task-specific input text and output triples. By taking the task-specific input, we enable a task-agnostic translation by leveraging the latent knowledge that a pre-trained language model has about the task. We further demonstrate that a simple pre-training task of predicting which relational information corresponds to which input text is an effective way to produce task-specific outputs. This enables the zero-shot transfer of our framework to downstream tasks. We study the zero-shot performance of this framework on open information extraction (OIE2016, NYT, WEB, PENN), relation classification (FewRel and TACRED), and factual probe (Google-RE and T-REx). The model transfers…

Semi-supervised Learning over Heterogeneous Information Networks by Ensemble of Meta-graph Guided Random Walks

He Jiang, Yangqiu Song, Chenguang Wang +2 more

Artificial Intelligence · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · 28 Jul 2017 · cited 44× · DOI ↗ · more like this ·

Heterogeneous information networks (HINs) is a general representation of many real world applications. The difference between HIN and traditional homogeneous graphs is that the nodes and edges in HIN are with types. Then in the many applications, we need to consider the types to make the approach more semantically meaningful. For the applications that annotation is expensive, on natural way is to consider semi-supervised learning over HIN. In this paper, we present a semi-supervised learning algorithm constrained by the types of HINs. We first decompose the original HIN into several semantically meaningful sub-graphs based the meta-graphs composed of entity and relation types. Then we perform random walk over the sub-graphs to propagate the labels from labeled data to unlabeled data. After we obtain all the labels propagated by different trials of random walk guided by meta-graphs, we…

Multiaperture Visual Velocity Measurement Method Based on Biomimetic Compound-Eye for UAVs

Chong Shen, Xin Zhao, Xindong Wu +4 more

Computer Vision and Pattern Recognition · IEEE Internet of Things Journal · 17 Oct 2023 · cited 41× · DOI ↗ · more like this ·

Autonomous velocity measurement technology based on optical flow plays an important role in applications of the Internet of Things. However, robust velocity measurement results need to provide a robust optical flow field and distance to the ground, especially for complex ground scenes. To solve this problem, this article proposes a visual velocity measurement method based on a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$3\times3$ </tex-math></inline-formula> camera-array multiaperture biomimetic compound-eye imaging system. The multiaperture optical flow field is first obtained from compound-eye through multiscale analysis and Bayes threshold processing. The real distance to the ground is then obtained by extracting the disparity information of nine apertures for adaptive depth estimation. The…

A hybrid decision support system with golden cut and bipolar q-ROFSs for evaluating the risk-based strategic priorities of fintech lending for clean energy projects

Qilong Wan, Xiaodong Miao, Chenguang Wang +2 more

Artificial Intelligence · Financial Innovation · 7 Jan 2023 · cited 41× · DOI ↗ · more like this ·

In the last decade, the risk evaluation and the investment decision are among the most prominent issues of efficient project management. Especially, the innovative financial sources could have some specific risk appetite due to the increasing return of investment. Hence, it is important to uncover the risk factors of fintech investments and investigate the possible impacts with an integrated approach to the strategic priorities of fintech lending. Accordingly, this study aims to analyze a unique risk set and the strategic priorities of fintech lending for clean energy projects. The most important contributions to the literature can be listed as to construct an impact-direction map of risk-based strategic priorities for fintech lending in clean energy projects and to measure the possible influences by using a hybrid decision making system with golden cut and bipolar q-rung orthopair…

Propensity score-integrated composite likelihood approach for incorporating real-world evidence in single-arm clinical studies

Chenguang Wang, Nelson Lu, Wei-Chen Chen +4 more

Statistics and Probability · Journal of Biopharmaceutical Statistics · 10 Nov 2019 · cited 38× · DOI ↗ · more like this ·

In medical product development, there has been an increased interest in utilizing real-world data which have become abundant with recent advances in biomedical science, information technology, and engineering. High-quality real-world data may be analyzed to generate real-world evidence that can be utilized in the regulatory and healthcare decision-making. In this paper, we consider the case in which a single-arm clinical study, viewed as the primary data source, is supplemented with patients from a real-world data source containing both clinical outcome and covariate data at the patient-level. Propensity score methodology is used to identify real-world data patients that are similar to those in the single-arm study in terms of the baseline characteristics, and to stratify these patients into strata based on the proximity of the propensity scores. In each stratum, a composite likelihood…

Research on DDoS Attacks Detection Based on RDF-SVM

Chenguang Wang, Jing Zheng, Xiaoyong Li

Computer Networks and Communications · October 2017 · cited 36× · DOI ↗ · more like this ·

DDoS attacks bring huge threaten to network, how to effectively detect DDoS is a hot topic of information security. Currently, there are some methods designed to detect DDoS attacks, but the detection rate of them is low. Moreover, DDoS detection is easily misled by flash crowd traffic. In this paper, a new method to detect DDoS attacks based on RDF-SVM algorithm is proposed. By considering the importance of feature selection in DDoS attacks detection, the RDF-SVM algorithm is designed to exploit random forest to compute the feature importance and SVM to rescreen the features, which will prevent from removing features mistakenly. Finally, an optimal feature subset is obtained, which will reach a higher detection rate and recall rate. In this paper, two kinds of datasets are used to train and test. The experimental result shows that the RDF-SVM algorithm can select the optimal feature…

Information Monitoring and Adaptive Information Fusion of Multisource Fusion Navigation Systems in Complex Environments

Huijun Zhao, Jun Liu, Xuemei Chen +5 more

Artificial Intelligence · IEEE Internet of Things Journal · 22 Apr 2024 · cited 35× · DOI ↗ · more like this ·

Accurately obtaining the navigation information of the device is crucial for realizing various emerging Internet of Things (IoT) applications, and a multi-source fusion navigation system is the key to achieving this goal. A distributed integrated inertial navigation system (INS), polarization compass (PC), and geomagnetic compass (MAG) enhanced direction approach is presented to improve the accuracy and robustness of the multisource fusion navigation system in complex environments. To estimate the time-varying measurement noise covariance in a nonlinear multi-source fusion navigation system, the traditional federated Kalman filter (FKF) is improved. In the FKF framework, the third-order spherical radial cubature rule and variational Bayesian theory are introduced, and a variational Bayesian federated cubature Kalman filter (VBFCKF) is proposed. Furthermore, a distributed information…