arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Doug Beeferman

23 papers · 1994–2024

1994: 11995: 01996: 11997: 21998: 21999: 12000: 02001: 02002: 12003: 02004: 02005: 02006: 02007: 02008: 02009: 02010: 12011: 12012: 02013: 12014: 02015: 02016: 02017: 02018: 02019: 12020: 02021: 42022: 22023: 32024: 2
1994peak 4 in 20212024

Frequent co-authors

Papers

showing 1–23 of 23 RSS

Text Segmentation Using Exponential Models

Doug Beeferman, Adam Berger, John Lafferty

cmp-lg · · cited 157× · arXiv ↗ · more like this ·

This paper introduces a new statistical approach to partitioning text automatically into coherent segments. Our approach enlists both short-range and long-range language models to help it sniff out likely sites of topic changes in text. To aid its search, the system consults a set of simple lexical hints it has learned to associate with the presence of boundaries through inspection of a large corpus of annotated data. We also propose a new probabilistically motivated error metric for use by the natural language processing and information retrieval communities, intended to supersede precision and recall for appraising segmentation algorithms. Qualitative assessment of our algorithm as well as evaluation using this new metric demonstrate the effectiveness of our approach in two very different domains, Wall Street Journal articles and the TDT Corpus, a collection of newswire articles and…

A Model of Lexical Attraction and Repulsion

Doug Beeferman, Adam Berger, John Lafferty

cmp-lg · · cited 110× · arXiv ↗ · more like this ·

This paper introduces new methods based on exponential families for modeling the correlations between words in text and speech. While previous work assumed the effects of word co-occurrence statistics to be constant over a window of several hundred words, we show that their influence is nonstationary on a much smaller time scale. Empirical data drawn from English and Japanese text, as well as conversational speech, reveals that the ``attraction'' between words decays exponentially, while stylistic and syntactic contraints create a ``repulsion'' between words that discourages close co-occurrence. We show that these characteristics are well described by simple mixture models based on two-stage exponential distributions which can be trained using the EM algorithm. The resulting distance distributions can then be incorporated as penalizing features in an exponential language model.

Cyberpunc: a lightweight punctuation annotation system for speech

Doug Beeferman, Adam Berger, John Lafferty

Artificial Intelligence · · cited 85× · DOI ↗ · more like this ·

This paper describes a lightweight method for the automatic insertion of intra-sentence punctuation into text. Despite the intuition that pauses in an acoustic stream are a positive indicator for some types of punctuation, this work will demonstrate the feasibility of a system which relies solely on lexical information. Besides its potential role in a speech recognition system, such a system could serve equally well in non-speech applications such as automatic grammar correction in a word processor and parsing of spoken text. After describing the design of a punctuation-restoration system, which relies on a trigram language model and a straightforward application of the Viterbi algorithm, we summarize results, both quantitative and subjective, of the performance and behavior of a prototype system.

Improved topic-dependent language modeling using information retrieval techniques

M. Mahajan, Doug Beeferman, Xuedong Huang

Artificial Intelligence · · cited 55× · DOI ↗ · more like this ·

N-gram language models are frequently used by the speech recognition systems to constrain and guide the search. N-gram models use only the last N-1 words to predict the next word. Typical values of N that are used range from 2-4. N-gram language models thus lack the long-term context information. We show that the predictive power of the N-gram language models can be improved by using long-term context information about the topic of discussion. We use information retrieval techniques to generalize the available context information for topic-dependent language modeling. We demonstrate the effectiveness of this technique by performing experiments on the Wall Street Journal text corpus, which is a relatively difficult task for topic-dependent language modeling since the text is relatively homogeneous. The proposed method can reduce the perplexity of the baseline language model by 37%…

Lexical Discovery with an Enriched Semantic Network.

Doug Beeferman

Artificial Intelligence · International Conference on Computational Linguistics · · cited 30× · OpenAlex ↗ · more like this ·

The study of lexical semantics has produced a systematic analysis of binary relationships between content words that has greatly benefited lexical search tools and natural language processing algorithms. We first introduce a database system called FreeNet that facilitates the description and exploration of finite binary relations. We then describe the design and implementation of Lexical FreeNet, a semantic network that mixes WordNet-derived semantic relations with data-derived and phonetically-derived relations. We discuss how Lexical FreeNet has aided in lexical discovery, the pursuit of linguistic and factual knowledge by the computer-aided exploration of lexical relations. 1 Motivation This paper discusses Lexical FreeNet, a database system designed to enhance lexical discovery. By this we mean the pursuit of linguistic and factual knowledge with the computer-aided exploration of…

Annotating the Tweebank Corpus on Named Entity Recognition and Building NLP Models for Social Media Analysis

Hang Jiang, Yining Hua, Doug Beeferman +1 more

cs.CL · · cited 28× · arXiv ↗ · more like this ·

Social media data such as Twitter messages ("tweets") pose a particular challenge to NLP systems because of their short, noisy, and colloquial nature. Tasks such as Named Entity Recognition (NER) and syntactic parsing require highly domain-matched training data for good performance. To date, there is no complete training corpus for both NER and syntactic analysis (e.g., part of speech tagging, dependency parsing) of tweets. While there are some publicly available annotated NLP datasets of tweets, they are only designed for individual tasks. In this study, we aim to create Tweebank-NER, an English NER corpus based on Tweebank V2 (TB2), train state-of-the-art (SOTA) Tweet NLP models on TB2, and release an NLP pipeline called Twitter-Stanza. We annotate named entities in TB2 using Amazon Mechanical Turk and measure the quality of our annotations. We train the Stanza pipeline on TB2 and…

Redrawing attendance boundaries to promote racial and ethnic diversity in elementary schools

Nabeel Gillani, Doug Beeferman, Christine Vega-Pourheydarian +3 more

cs.CY · · cited 20× · arXiv ↗ · more like this ·

Most US school districts draw "attendance boundaries" to define catchment areas that assign students to schools near their homes, often recapitulating neighborhood demographic segregation in schools. Focusing on elementary schools, we ask: how much might we reduce school segregation by redrawing attendance boundaries? Combining parent preference data with methods from combinatorial optimization, we simulate alternative boundaries for 98 US school districts serving over 3 million elementary-aged students, minimizing White/non-White segregation while mitigating changes to travel times and school sizes. Across districts, we observe a median 14% relative decrease in segregation, which we estimate would require approximately 20\% of students to switch schools and, surprisingly, a slight reduction in travel times. We release a public dashboard depicting these alternative boundaries…

Engaging Politically Diverse Audiences on Social Media

Martin Saveski, Doug Beeferman, David McClure +1 more

cs.SI · · cited 18× · arXiv ↗ · more like this ·

We study how political polarization is reflected in the social media posts used by media outlets to promote their content online. In particular, we track the Twitter posts of several media outlets over the course of more than three years (566K tweets), and the engagement with these tweets from other users (104M retweets), modeling the relationship between the tweet text and the political diversity of the audience. We build a tool that integrates our model and helps journalists craft tweets that are engaging to a politically diverse audience, guided by the model predictions. To test the real-world impact of the tool, we partner with the PBS documentary series Frontline and run a series of advertising experiments on Twitter. We find that in seven out of the ten experiments, the tweets selected by our model were indeed engaging to a more politically diverse audience, illustrating the…

RadioTalk: a large-scale corpus of talk radio transcripts

Doug Beeferman, William Brannon, Deb Roy

cs.CL · · cited 16× · arXiv ↗ · more like this ·

We introduce RadioTalk, a corpus of speech recognition transcripts sampled from talk radio broadcasts in the United States between October of 2018 and March of 2019. The corpus is intended for use by researchers in the fields of natural language processing, conversational analysis, and the social sciences. The corpus encompasses approximately 2.8 billion words of automatically transcribed speech from 284,000 hours of radio, together with metadata about the speech, such as geographical location, speaker turn boundaries, gender, and radio program information. In this paper we summarize why and how we prepared the corpus, give some descriptive statistics on stations, shows and speakers, and carry out a few high-level analyses.

Unsupervised testing strategies for ASR

Brian Strope, Doug Beeferman, Alexander Gruenstein +1 more

Artificial Intelligence · · cited 12× · DOI ↗ · more like this ·

This paper describes unsupervised strategies for estimating relative accuracy differences between acoustic models or language models used for automatic speech recognition. To test acoustic models, the approach extends ideas used for unsupervised discriminative training to include a more explicit validation on held out data. To test language models, we use a dual interpretation of the same process, this time allowing us to measure differences by exploiting expected ‘truth gradients’ between strong and weak acoustic models. The paper shows correlations between supervised and unsupervised measures across a range of acoustic model and language model variations. We also use unsupervised tests to assess the non-stationary nature of mobile speech input. Index Terms: speech recognition, unsupervised testing, nonstationary distributions

Parents’ Online School Reviews Reflect Several Racial and Socioeconomic Disparities in K–12 Education

Nabeel Gillani, Eric Chu, Doug Beeferman +2 more

Education · AERA Open · · cited 10× · DOI ↗ · more like this ·

Parents often select schools by relying on subjective assessments of quality made by other parents, which are increasingly becoming available through written reviews on school ratings websites. To identify relationships between review content and school quality, we apply recent advances in natural language processing to nearly half a million parent reviews posted for more than 50,000 publicly funded U.S. K–12 schools on a popular ratings website. We find: (1) schools in urban areas and those serving affluent families are more likely to receive reviews, (2) review language correlates with standardized test scores—which generally track race and family income—but not school effectiveness, measured by how much students improve in their test scores over time, and (3) the linguistics of reviews reveal several racial and income-based disparities in K–12 education. These findings suggest that…

FeedbackMap: a tool for making sense of open-ended survey responses

Doug Beeferman, Nabeel Gillani

cs.CL · · cited 9× · arXiv ↗ · more like this ·

Analyzing open-ended survey responses is a crucial yet challenging task for social scientists, non-profit organizations, and educational institutions, as they often face the trade-off between obtaining rich data and the burden of reading and coding textual responses. This demo introduces FeedbackMap, a web-based tool that uses natural language processing techniques to facilitate the analysis of open-ended survey responses. FeedbackMap lets researchers generate summaries at multiple levels, identify interesting response examples, and visualize the response space through embeddings. We discuss the importance of examining survey results from multiple perspectives and the potential biases introduced by summarization methods, emphasizing the need for critical evaluation of the representation and omission of respondent voices.

CommunityLM: Probing Partisan Worldviews from Language Models

Hang Jiang, Doug Beeferman, Brandon Roy +1 more

cs.SI · · cited 9× · arXiv ↗ · more like this ·

As political attitudes have diverged ideologically in the United States, political speech has diverged lingusitically. The ever-widening polarization between the US political parties is accelerated by an erosion of mutual understanding between them. We aim to make these communities more comprehensible to each other with a framework that probes community-specific responses to the same survey questions using community language models CommunityLM. In our framework we identify committed partisan members for each community on Twitter and fine-tune LMs on the tweets authored by them. We then assess the worldviews of the two groups using prompt-based probing of their corresponding LMs, with prompts that elicit opinions about public figures and groups surveyed by the American National Election Studies (ANES) 2020 Exploratory Testing Survey. We compare the responses generated by the LMs to the…

The rhythm of lexical stress in prose

Doug Beeferman

Artificial Intelligence · The COCOON platform (University of Paris) · · cited 3× · DOI ↗ · more like this ·

"Prose rhythm" is a widely observed but scarcely quantified phenomenon. We describe an information-theoretic model for measuring the regularity of lexical stress in English texts, and use it in combination with trigram language models to demonstrate a relationship between the probability of word sequences in English and the amount of rhythm present in them. We find that the stream of lexical stress in text from the Wall Street Journal has an entropy rate of less than 0.75 bits per syllable for common sentences. We observe that the average number of syllables per word is greater for rarer word sequences, and to normalize for this effect we run control experiments to show that the choice of word order contributes significantly to stress regularity, and increasingly with lexical probability.

From CMU Sphinx-II to Microsoft Whisper: Making Speech Recognition Usable

Fil Alleva, Doug Beeferman, Milind Mahajan +2 more

Artificial Intelligence · · cited 3× · OpenAlex ↗ · more like this ·

In this paper, we first review Sphinx-II, a large-vocabulary speaker-independent continuous speech recognition system developed at CMU. We summarize techniques that helped Sphinx-II achieve the state-of-the-art large-vocabulary continuous speech recognition performance. We then review Whisper, a system we developed here at Microsoft Research. We focus on recognition accuracy, efficiency and usability issues, which are critical to the success of commercial speech applications. Whisper has significantly improved Sphinx-II’s performance in these three areas.

AudienceView: AI-Assisted Interpretation of Audience Feedback in Journalism

William Brannon, Doug Beeferman, Hang Jiang +2 more

cs.HC · · cited 1× · arXiv ↗ · more like this ·

Understanding and making use of audience feedback is important but difficult for journalists, who now face an impractically large volume of audience comments online. We introduce AudienceView, an online tool to help journalists categorize and interpret this feedback by leveraging large language models (LLMs). AudienceView identifies themes and topics, connects them back to specific comments, provides ways to visualize the sentiment and distribution of the comments, and helps users develop ideas for subsequent reporting projects. We consider how such tools can be useful in a journalist's workflow, and emphasize the importance of contextual awareness and human judgment.

Bridging Dictionary: AI-Generated Dictionary of Partisan Language Use

Hang Jiang, Doug Beeferman, William Brannon +2 more

cs.HC · · cited 1× · arXiv ↗ · more like this ·

Words often carry different meanings for people from diverse backgrounds. Today's era of social polarization demands that we choose words carefully to prevent miscommunication, especially in political communication and journalism. To address this issue, we introduce the Bridging Dictionary, an interactive tool designed to illuminate how words are perceived by people with different political views. The Bridging Dictionary includes a static, printable document featuring 796 terms with summaries generated by a large language model. These summaries highlight how the terms are used distinctively by Republicans and Democrats. Additionally, the Bridging Dictionary offers an interactive interface that lets users explore selected words, visualizing their frequency, sentiment, summaries, and examples across political divides. We present a use case for journalists and emphasize the importance of…

Topic Detection and Tracking with Time-Aware Document Embeddings

Hang Jiang, Doug Beeferman, Weiquan Mao +1 more

cs.CL · · cited 1× · arXiv ↗ · more like this ·

The time at which a message is communicated is a vital piece of metadata in many real-world natural language processing tasks such as Topic Detection and Tracking (TDT). TDT systems aim to cluster a corpus of news articles by event, and in that context, stories that describe the same event are likely to have been written at around the same time. Prior work on time modeling for TDT takes this into account, but does not well capture how time interacts with the semantic nature of the event. For example, stories about a tropical storm are likely to be written within a short time interval, while stories about a movie release may appear over weeks or months. In our work, we design a neural method that fuses temporal and textual information into a single representation of news documents for event detection. We fine-tune these time-aware document embeddings with a triplet loss architecture…

Topic-time Heatmaps for Human-in-the-loop Topic Detection and Tracking

Doug Beeferman, Hang Jiang

cs.IR · · cited 1× · arXiv ↗ · more like this ·

The essential task of Topic Detection and Tracking (TDT) is to organize a collection of news media into clusters of stories that pertain to the same real-world event. To apply TDT models to practical applications such as search engines and discovery tools, human guidance is needed to pin down the scope of an "event" for the corpus of interest. In this work in progress, we explore a human-in-the-loop method that helps users iteratively fine-tune TDT algorithms so that both the algorithms and the users themselves better understand the nature of the events. We generate a visual overview of the entire corpus, allowing the user to select regions of interest from the overview, and then ask a series of questions to affirm (or reject) that the selected documents belong to the same event. The answers to these questions supplement the training data for the event similarity model that underlies…

All a-board: sharing educational data science research with school districts

Nabeel Gillani, Doug Beeferman, Cassandra Overney +2 more

cs.CY · · cited 0× · arXiv ↗ · more like this ·

Educational data scientists often conduct research with the hopes of translating findings into lasting change through policy, civil society, or other channels. However, the bridge from research to practice can be fraught with sociopolitical frictions that impede, or altogether block, such translations -- especially when they are contentious or otherwise difficult to achieve. Focusing on one entrenched educational equity issue in US public schools -- racial and ethnic segregation -- we conduct randomized email outreach experiments and surveys to explore how local school districts respond to algorithmically-generated school catchment areas ("attendance boundaries") designed to foster more diverse and integrated schools. Cold email outreach to approximately 4,320 elected school board members across over 800 school districts informing them of potential boundary changes reveals a large…

INTERSPEECH 2011 Unsupervised Testing Strategies for ASR

Brian Strope, Doug Beeferman, Er Gruenstein +2 more

Artificial Intelligence · · cited 0× · OpenAlex ↗ · more like this ·

This paper describes unsupervised strategies for estimating relative accuracy differences between acoustic models or language models used for automatic speech recognition. To test acoustic models, the approach extends ideas used for unsupervised discriminative training to include a more explicit validation on held out data. To test language models, we use a dual interpretation of the same process, this time allowing us to measure differences by exploiting expected ‘truth gradients ’ between strong and weak acoustic models. The paper shows correlations between supervised and unsupervised measures across a range of acoustic model and language model variations. We also use unsupervised tests to assess the non-stationary nature of mobile speech input. Index Terms: speech recognition, unsupervised testing, nonstationary distributions

Lexical Discovery with an Enriched Semantic Network

Doug Beeferman School, Doug Beeferman

Artificial Intelligence · · cited 0× · OpenAlex ↗ · more like this ·

The study of lexical semantics has produced a systematic analysis of binary relationships between content words that has greatly benefited lexical search tools and natural language processing algorithms. We first introduce a database system called FreeNet that facilitates the description and exploration of finite binary relations. We then describe the design and implementation of Lexical FreeNet, a semantic network that mixes WordNet-derived semantic re- lations with data-derived and phonetically-derived relations. We discuss how Lexical FreeNet has aided in lexical discovery, the pursuit of linguistic and factual knowledge by the computer-aided exploration of lexical relations.