arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Benjamin Roth

114 papers · 2009–2026

2009: 12010: 42011: 22012: 42013: 52014: 82015: 52016: 32017: 22018: 102019: 32020: 42021: 92022: 82023: 82024: 142025: 132026: 11
2009peak 14 in 20242026

Frequent co-authors

Papers

showing 1–25 of 114 RSS

Compositional Vector Space Models for Knowledge Base Completion

Arvind Neelakantan, Benjamin Roth, Andrew McCallum

cs.CL · · cited 302× · arXiv ↗ · more like this ·

Knowledge base (KB) completion adds new facts to a KB by making inferences from existing facts, for example by inferring with high likelihood nationality(X,Y) from bornIn(X,Y). Most previous methods infer simple one-hop relational synonyms like this, or use as evidence a multi-hop relational path treated as an atomic feature, like bornIn(X,Z) -> containedIn(Z,Y). This paper presents an approach that reasons about conjunctions of multi-hop relations non-atomically, composing the implications of a path using a recursive neural network (RNN) that takes as inputs vector embeddings of the binary relation in the path. Not only does this allow us to generalize to paths unseen at training time, but also, with a single high-capacity RNN, to predict new relation types not seen when the compositional model was trained (zero-shot learning). We assemble a new dataset of over 52M relational triples…

A survey on the role of negation in sentiment analysis

Michael Wiegand, Alexandra Balahur, Benjamin Roth +2 more

Artificial Intelligence · Publication Server of the Institute for German Language (Institute for German Language) · · cited 215× · OpenAlex ↗ · more like this ·

This paper presents a survey on the role of negation in sentiment analysis. Negation is a very common linguistic construction that affects polarity and, therefore, needs to be taken into consideration in sentiment analysis. We will present various computational approaches modeling negation in sentiment analysis. We will, in particular, focus on aspects such as level of representation used for sentiment analysis, negation word detection and scope of negation. We will also discuss limits and challenges of negation modeling on that task.

Joint Aspect and Polarity Classification for Aspect-based Sentiment Analysis with End-to-End Neural Networks

Martin Schmitt, Simon Steinheber, Konrad Schreiber +1 more

cs.CL · · cited 124× · arXiv ↗ · more like this ·

In this work, we propose a new model for aspect-based sentiment analysis. In contrast to previous approaches, we jointly model the detection of aspects and the classification of their polarity in an end-to-end trainable neural network. We conduct experiments with different neural architectures and word representations on the recent GermEval 2017 dataset. We were able to show considerable performance gains by using the joint modeling approach in all settings compared to pipeline approaches. The combination of a convolutional neural network and fasttext embeddings outperformed the best submission of the shared task in 2017, establishing a new state of the art.

Multilingual Relation Extraction using Compositional Universal Schema

Patrick Verga, David Belanger, Emma Strubell +2 more

cs.CL · · cited 99× · arXiv ↗ · more like this ·

Universal schema builds a knowledge base (KB) of entities and relations by jointly embedding all relation types from input KBs as well as textual patterns expressing relations from raw text. In most previous applications of universal schema, each textual pattern is represented as a single embedding, preventing generalization to unseen patterns. Recent work employs a neural network to capture patterns' compositional semantics, providing generalization to all possible input text. In response, this paper introduces significant further improvements to the coverage and flexibility of universal schema relation extraction: predictions for entities unseen in training and multilingual transfer learning to domains with no annotation. We evaluate our model through extensive experiments on the English and Spanish TAC KBP benchmark, outperforming the top system from TAC 2013 slot-filling using no…

Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement

Nina Poerner, Hinrich Schütze, Benjamin Roth

Artificial Intelligence · Open access LMU (Ludwid Maxmilian's Universitat Munchen) · · cited 85× · DOI ↗ · more like this ·

The behavior of deep neural networks (DNNs) is hard to understand. This makes it necessary to explore post hoc explanation methods. We conduct the first comprehensive evaluation of explanation methods for NLP. To this end, we design two novel evaluation paradigms that cover two important classes of NLP problems: small context and large context problems. Both paradigms require no manual annotation and are therefore broadly applicable. We also introduce LIMSSE, an explanation method inspired by LIME that is designed for NLP. We show empirically that LIMSSE, LRP and DeepLIFT are the most effective explanation methods and recommend them for explaining DNNs in NLP.

Topic Models for Word Sense Disambiguation and Token-Based Idiom Detection

Linlin Li, Benjamin Roth, Caroline Sporleder

Artificial Intelligence · Meeting of the Association for Computational Linguistics · · cited 67× · OpenAlex ↗ · more like this ·

This paper presents a probabilistic model for sense disambiguation which chooses the best sense based on the conditional probability of sense paraphrases given a context. We use a topic model to decompose this conditional probability into two conditional probabilities with latent variables. We propose three different instantiations of the model for solving sense disambiguation problems with different degrees of resource availability. The proposed models are tested on three different tasks: coarse-grained word sense disambiguation, fine-grained word sense disambiguation, and detection of literal vs. nonliteral usages of potentially idiomatic expressions. In all three cases, we outperform state-of-the-art systems either quantitatively or statistically significantly. 1

Effective Slot Filling Based on Shallow Distant Supervision Methods

Benjamin Roth, Tassilo Barth, Michael Wiegand +2 more

cs.CL · · cited 61× · arXiv ↗ · more like this ·

Spoken Language Systems at Saarland University (LSV) participated this year with 5 runs at the TAC KBP English slot filling track. Effective algorithms for all parts of the pipeline, from document retrieval to relation prediction and response post-processing, are bundled in a modular end-to-end relation extraction system called RelationFactory. The main run solely focuses on shallow techniques and achieved significant improvements over LSV's last year's system, while using the same training data and patterns. Improvements mainly have been obtained by a feature representation focusing on surface skip n-grams and improved scoring for extracted distant supervision patterns. Important factors for effective extraction are the training and tuning scheme for distant supervision classifiers, and the query expansion by a translation model based on Wikipedia links. In the TAC KBP 2013 English…

A survey of noise reduction methods for distant supervision

Benjamin Roth, Tassilo Barth, Michael Wiegand +1 more

Artificial Intelligence · · cited 60× · DOI ↗ · more like this ·

We survey recent approaches to noise reduction in distant supervision learning for relation extraction. We group them according to the principles they are based on: at-least-one constraints, topic-based models, or pattern correlations. Besides describing them, we illustrate the fundamental differences and attempt to give an outlook to potentially fruitful further research. In addition, we identify related work in sentiment analysis which could profit from approaches to noise reduction.

Compositional Vector Space Models for Knowledge Base Inference.

Arvind Neelakantan, Benjamin Roth, Andrew McCallum

Artificial Intelligence · National Conference on Artificial Intelligence · · cited 59× · OpenAlex ↗ · more like this ·

Knowledge base (KB) completion adds new facts to a KB by making inferences from existing facts, for example by infer-ring with high likelihood nationality(X,Y) from bornIn(X,Y). Most previous methods infer simple one-hop relational synonyms like this, or use as evidence a multi-hop re-lational path treated as an atomic feature, like bornIn(X,Z) → containedIn(Z,Y). This paper presents an approach that reasons about conjunctions of multi-hop relations non-atomically, composing the implica-tions of a path using a recurrent neural network (RNN) that takes as inputs vec-tor embeddings of the binary relation in the path. Not only does this allow us to generalize to paths unseen at training time, but also, with a single high-capacity RNN, to predict new relation types not seen when the compositional model was trained (zero-shot learning). We assem-ble a new dataset of over 52M relational…

Comparing Convolutional Neural Networks to Traditional Models for Slot Filling

Heike Adel, Benjamin Roth, Hinrich Schütze

cs.CL · · cited 46× · arXiv ↗ · more like this ·

We address relation classification in the context of slot filling, the task of finding and evaluating fillers like "Steve Jobs" for the slot X in "X founded Apple". We propose a convolutional neural network which splits the input sentence into three parts according to the relation arguments and compare it to state-of-the-art and traditional approaches of relation classification. Finally, we combine different methods and show that the combination is better than individual approaches. We also analyze the effect of genre differences on performance.

School Social Workers as <i>Nepantleras</i> in Equity Work for Immigrant Students: A Conceptual Exploration

Sophia Rodriguez, Benjamin Roth, Leticia Villarreal Sosa

Education · Social Service Review · · cited 39× · DOI ↗ · more like this ·

This article investigates the critical role school social workers play in promoting equity and advocating for immigrant students in K–12 settings. We explore and conceptualize the notion of nepantlera from the work of Gloria Anzaldúa and draw from qualitative data to show how school social workers function as nepantleras in service and advocacy roles for immigrant students. We further argue that the school settings in which these services take place should be understood as borderlands. Throughout the article, we explore the following questions: What is a paradigm for school social worker practice with immigrant students that is informed by the concept of nepantlera? How do we begin to consider such a paradigm in a way that honors the complexity of Anzaldúa’s work? While pursuing these questions we present qualitative data from a national sample of school workers. We offer implications…

Combining Generative and Discriminative Model Scores for Distant Supervision

Benjamin Roth, Dietrich Klakow

Artificial Intelligence · · cited 36× · DOI ↗ · more like this ·

Distant supervision is a scheme to generate noisy training data for relation extraction by aligning entities of a knowledge base with text. In this work we combine the output of a discriminative at-least-one learner with that of a generative hierarchical topic model to re-duce the noise in distant supervision data. The combination significantly increases the rank-ing quality of extracted facts and achieves state-of-the-art extraction performance in an end-to-end setting. A simple linear interpo-lation of the model scores performs better than a parameter-free scheme based on non-dominated sorting. 1

Perceived advantages: the influence of urban and suburban neighbourhood context on the socialization and adaptation of Mexican immigrant young men

Florian Sichling, Benjamin Roth

Sociology and Political Science · Ethnic and Racial Studies · · cited 34× · DOI ↗ · more like this ·

Neighbourhoods are important contexts for the socialization and adaptation of immigrant youth. They provide resources and opportunities for positive interactions with peers and non-family adults. But neighbourhoods differ systemically in their demographic composition and the type and quality of resources they offer young people. In the US, there is an implicit assumption equating suburban neighbourhoods with better schools, more jobs and higher quality housing compared to urban neighbourhoods. There is however, little explicit empirical evidence of how such differences may shape the experience of immigrant youth. This gap is concerning in light of recent trends of immigrants to move directly to the suburbs. The first part of the paper reviews the literatures on immigrant adaptation and neighbourhood effects. Drawing on two qualitative studies, the second part of the paper illustrates…

Interpretable Question Answering on Knowledge Bases and Text

Alona Sydorova, Nina Poerner, Benjamin Roth

cs.CL · · cited 29× · arXiv ↗ · more like this ·

Interpretability of machine learning (ML) models becomes more relevant with their increasing adoption. In this work, we address the interpretability of ML based question answering (QA) models on a combination of knowledge bases (KB) and text documents. We adapt post hoc explanation methods such as LIME and input perturbation (IP) and compare them with the self-explanatory attention mechanism of the model. For this purpose, we propose an automatic evaluation paradigm for explanation methods in the context of QA. We also conduct a study with human annotators to evaluate whether explanations help them identify better QA models. Our results suggest that IP provides better explanations than LIME or attention, according to both automatic and human evaluation. We obtain the same ranking of methods in both experiments, which supports the validity of our automatic evaluation paradigm.

Interpretable Textual Neuron Representations for NLP

Nina Poerner, Benjamin Roth, Hinrich Schütze

cs.CL · · cited 28× · arXiv ↗ · more like this ·

Input optimization methods, such as Google Deep Dream, create interpretable representations of neurons for computer vision DNNs. We propose and evaluate ways of transferring this technology to NLP. Our results suggest that gradient ascent with a gumbel softmax layer produces n-gram representations that outperform naive corpus search in terms of target neuron activation. The representations highlight differences in syntax awareness between the language and visual models of the Imaginet architecture.

Specification Overfitting in Artificial Intelligence

Benjamin Roth, Pedro Henrique Luz de Araujo, Yuxi Xia +2 more

cs.AI · · cited 20× · arXiv ↗ · more like this ·

Machine learning (ML) and artificial intelligence (AI) approaches are often criticized for their inherent bias and for their lack of control, accountability, and transparency. Consequently, regulatory bodies struggle with containing this technology's potential negative side effects. High-level requirements such as fairness and robustness need to be formalized into concrete specification metrics, imperfect proxies that capture isolated aspects of the underlying requirements. Given possible trade-offs between different metrics and their vulnerability to over-optimization, integrating specification metrics in system development processes is not trivial. This paper defines specification overfitting, a scenario where systems focus excessively on specified metrics to the detriment of high-level requirements and task performance. We present an extensive literature survey to categorize how…

Feature-based models for improving the quality of noisy training data for relation extraction

Benjamin Roth, Dietrich Klakow

Artificial Intelligence · · cited 15× · DOI ↗ · more like this ·

Supervised relation extraction from text relies on annotated data. Distant supervision is a scheme to obtain noisy training data by using a knowledge base of relational tuples as the ground truth and finding entity pair matches in a text corpus. We propose and evaluate two feature-based models for increasing the quality of distant supervision extraction patterns.

Domain adaptation for part-of-speech tagging of noisy user-generated text

Luisa März, Dietrich Trautmann, Benjamin Roth

cs.CL · · cited 14× · arXiv ↗ · more like this ·

The performance of a Part-of-speech (POS) tagger is highly dependent on the domain ofthe processed text, and for many domains there is no or only very little training data available. This work addresses the problem of POS tagging noisy user-generated text using a neural network. We propose an architecture that trains an out-of-domain model on a large newswire corpus, and transfers those weights by using them as a prior for a model trained on the target domain (a data-set of German Tweets) for which there is very little an-notations available. The neural network has two standard bidirectional LSTMs at its core. However, we find it crucial to also encode a set of task-specific features, and to obtain reliable (source-domain and target-domain) word representations. Experiments with different regularization techniques such as early stopping, dropout and fine-tuning the domain adaptation…

Towards Bootstrapping a Polarity Shifter Lexicon using Linguistic Features

Marc Schulder, Michael Wiegand, Josef Ruppenhofer +1 more

Artificial Intelligence · Publication Server of the Institute for German Language (Institute for German Language) · · cited 14× · DOI ↗ · more like this ·

We present a major step towards the creation of the first high-coverage lexicon of polarity shifters. In this work, we bootstrap a lexicon of verbs by exploiting various linguistic features. Polarity shifters, such as “abandon”, are similar to negations (e.g. “not”) in that they move the polarity of a phrase towards its inverse, as in “abandon all hope”. While there exist lists of negation words, creating comprehensive lists of polarity shifters is far more challenging due to their sheer number. On a sample of manually annotated verbs we examine a variety of linguistic features for this task. Then we build a supervised classifier to increase coverage. We show that this approach drastically reduces the annotation effort while ensuring a high-precision lexicon. We also show that our acquired knowledge of verbal polarity shifters improves phrase-level sentiment analysis.…

Advocating for Structural Change? Exploring the Advocacy Activities of Immigrant-Serving Organizations in an Unwelcoming Policy Context

Benjamin Roth, Seo Yeon Park, Breanne Grace

Public Administration · Advances in Social Work · · cited 12× · DOI ↗ · more like this ·

The growth of the immigrant population in the United States has prompted a recent increase in the number of restrictive immigration policies at the state and local levels. The literature on policy advocacy and social service organizations suggests that these local providers can engage in political activities that challenge the restrictive nature of these contexts. This qualitative study explored how immigrant-serving social service organizations engage in policy advocacy in a state with restrictive, anti-immigrant policies. In-depth interviews with directors of 50 service providers in South Carolina clearly indicate a tension between the need for policy advocacy and the risks associated with engaging in such activities. Fifty percent (50%) of the providers in our sample reported engaging in some form of policy advocacy. However, their policy advocacy activities were often indirect…

Automatic Food Categorization from Large Unlabeled Corpora and Its Impact on Relation Extraction

Michael Wiegand, Benjamin Roth, Dietrich Klakow

Artificial Intelligence · Publication Server of the Institute for German Language (Institute for German Language) · · cited 11× · DOI ↗ · more like this ·

We present a weakly-supervised induction method to assign semantic information to food items. We consider two tasks of categorizations being food-type classification and the distinction of whether a food item is composite or not. The categorizations are induced by a graph-based algorithm applied on a large unlabeled domain-specific corpus. We show that the usage of a domain-specific corpus is vital. We do not only outperform a manually designed open-domain ontology but also prove the usefulness of these categorizations in relation extraction, outperforming state-of-the-art features that include syntactic information and Brown clustering. Class 5 Manually Chosen Seeds (5-PROTO) 5 Hearst-Pattern Seeds (PAT-Top5)

Too Afraid to Stay: Measuring the Relationship between Criminal Victimization in Central America and the Intent to Migrate

Benjamin Roth, Amanda Huffman, Robert Brame

Sociology and Political Science · Crime & Delinquency · · cited 10× · DOI ↗ · more like this ·

Rising crime in Central America and on-going emigration have reinvigorated concerns about the relationship between criminal victimization and international migration. Analyzing data from a crime prevention intervention in this region, we examine measures related to criminal victimization to assess corresponding changes in the propensity for international migration. We find that individuals who reported a crime victimization were more likely to express their intention to migrate, but this pattern was less consistent for those who perceived their neighborhood to be unsafe. These descriptive patterns are important on their own, but this paper also advances our methodological understanding of how to address the problem of item nonresponse using a partial identification framework.

Cross-language retrieval using link-based language models

Benjamin Roth, Dietrich Klakow

Artificial Intelligence · · cited 9× · DOI ↗ · more like this ·

We propose a cross-language retrieval model that is solely based on Wikipedia as a training corpus. The main contributions of our work are: 1. A translation model based on linked text in Wikipedia and a term weighting method associated with it. 2. A combination scheme to interpolate the link translation model with retrieval based on Latent Dirichlet Allocation. On the CLEF 2000 data we achieve improvement with respect to the best German-English system at the bilingual track (non-significant) and improvement against a baseline based on machine translation (significant).

SepLL: Separating Latent Class Labels from Weak Supervision Noise

Andreas Stephan, Vasiliki Kougia, Benjamin Roth

cs.LG · · cited 8× · arXiv ↗ · more like this ·

In the weakly supervised learning paradigm, labeling functions automatically assign heuristic, often noisy, labels to data samples. In this work, we provide a method for learning from weak labels by separating two types of complementary information associated with the labeling functions: information related to the target label and information specific to one labeling function only. Both types of information are reflected to different degrees by all labeled instances. In contrast to previous works that aimed at correcting or removing wrongly labeled instances, we learn a branched deep model that uses all data as-is, but splits the labeling function information in the latent space. Specifically, we propose the end-to-end model SepLL which extends a transformer classifier by introducing a latent space for labeling function specific and task-specific information. The learning signal is only…