Author

Bryan McCann

65 papers · 2001–2025

2001peak 11 in 20192025

Frequent co-authors

Papers

showing 1–25 of 65 RSS

CTRL: A Conditional Transformer Language Model for Controllable Generation

Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney +2 more

cs.CL · 20 Sep 2019 · cited 1,493× · arXiv ↗ · more like this ·

Large-scale language models show promising text generation capabilities, but users cannot easily control particular aspects of the generated text. We release CTRL, a 1.63 billion-parameter conditional transformer language model, trained to condition on control codes that govern style, content, and task-specific behavior. Control codes were derived from structure that naturally co-occurs with raw text, preserving the advantages of unsupervised learning while providing more explicit control over text generation. These codes also allow CTRL to predict which parts of the training data are most likely given a sequence. This provides a potential method for analyzing large amounts of data via model-based source attribution. We have released multiple full-sized, pretrained versions of CTRL at https://github.com/salesforce/ctrl.

SummEval: Re-evaluating Summarization Evaluation

Alexander R. Fabbri, Wojciech Kryściński, Bryan McCann +3 more

cs.CL · July 2020 · cited 1,102× · arXiv ↗ · more like this ·

The scarcity of comprehensive up-to-date studies on evaluation metrics for text summarization and the lack of consensus regarding evaluation protocols continue to inhibit progress. We address the existing shortcomings of summarization evaluation methods along five dimensions: 1) we re-evaluate 14 automatic evaluation metrics in a comprehensive and consistent fashion using neural summarization model outputs along with expert and crowd-sourced human annotations, 2) we consistently benchmark 23 recent summarization models using the aforementioned automatic evaluation metrics, 3) we assemble the largest collection of summaries generated by models trained on the CNN/DailyMail news dataset and share it in a unified format, 4) we implement and share a toolkit that provides an extensible and unified API for evaluating summarization models across a broad range of automatic metrics, 5) we…

Evaluating the Factual Consistency of Abstractive Text Summarization

Wojciech Kryściński, Bryan McCann, Caiming Xiong +1 more

cs.CL · 28 Oct 2019 · cited 1,059× · arXiv ↗ · more like this ·

Currently used metrics for assessing summarization algorithms do not account for whether summaries are factually consistent with source documents. We propose a weakly-supervised, model-based approach for verifying factual consistency and identifying conflicts between source documents and a generated summary. Training data is generated by applying a series of rule-based transformations to the sentences of source documents. The factual consistency model is then trained jointly for three tasks: 1) identify whether sentences remain factually consistent after transformation, 2) extract a span in the source documents to support the consistency prediction, 3) extract a span in the summary sentence that is inconsistent if one exists. Transferring this model to summaries generated by several state-of-the art models reveals that this highly scalable approach substantially outperforms previous…

Learned in Translation: Contextualized Word Vectors

Bryan McCann, James Bradbury, Caiming Xiong +1 more

cs.CL · August 2017 · cited 946× · arXiv ↗ · more like this ·

Computer vision has benefited from initializing multiple deep layers with weights pretrained on large supervised training sets like ImageNet. Natural language processing (NLP) typically sees initialization of only the lowest layer of deep models with pretrained word vectors. In this paper, we use a deep LSTM encoder from an attentional sequence-to-sequence model trained for machine translation (MT) to contextualize word vectors. We show that adding these context vectors (CoVe) improves performance over using only unsupervised word and character vectors on a wide variety of common NLP tasks: sentiment analysis (SST, IMDb), question classification (TREC), entailment (SNLI), and question answering (SQuAD). For fine-grained sentiment analysis and entailment, CoVe improves performance of our baseline models to the state of the art.

The Natural Language Decathlon: Multitask Learning as Question Answering

Bryan McCann, Nitish Shirish Keskar, Caiming Xiong +1 more

cs.CL · 20 Jun 2018 · cited 667× · arXiv ↗ · more like this ·

Deep learning has improved performance on many natural language processing (NLP) tasks individually. However, general NLP models cannot emerge within a paradigm that focuses on the particularities of a single metric, dataset, and task. We introduce the Natural Language Decathlon (decaNLP), a challenge that spans ten tasks: question answering, machine translation, summarization, natural language inference, sentiment analysis, semantic role labeling, zero-shot relation extraction, goal-oriented dialogue, semantic parsing, and commonsense pronoun resolution. We cast all tasks as question answering over a context. Furthermore, we present a new Multitask Question Answering Network (MQAN) jointly learns all tasks in decaNLP without any task-specific modules or parameters in the multitask setting. MQAN shows improvements in transfer learning for machine translation and named entity…

Explain Yourself! Leveraging Language Models for Commonsense Reasoning

Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong +1 more

cs.CL · 6 Jun 2019 · cited 663× · arXiv ↗ · more like this ·

Deep learning models perform poorly on tasks that require commonsense reasoning, which often necessitates some form of world-knowledge or reasoning over information not immediately present in the input. We collect human explanations for commonsense reasoning in the form of natural language sequences and highlighted annotations in a new dataset called Common Sense Explanations (CoS-E). We use CoS-E to train language models to automatically generate explanations that can be used during training and inference in a novel Commonsense Auto-Generated Explanation (CAGE) framework. CAGE improves the state-of-the-art by 10% on the challenging CommonsenseQA task. We further study commonsense reasoning in DNNs using both human and auto-generated explanations including transfer to out-of-domain tasks. Empirical results indicate that we can effectively leverage language models for commonsense…

A Simple Language Model for Task-Oriented Dialogue

Ehsan Hosseini-Asl, Bryan McCann, Chien-Sheng Wu +2 more

cs.CL · May 2020 · cited 591× · arXiv ↗ · more like this ·

Task-oriented dialogue is often decomposed into three tasks: understanding user input, deciding actions, and generating a response. While such decomposition might suggest a dedicated model for each sub-task, we find a simple, unified approach leads to state-of-the-art performance on the MultiWOZ dataset. SimpleTOD is a simple approach to task-oriented dialogue that uses a single, causal language model trained on all sub-tasks recast as a single sequence prediction problem. This allows SimpleTOD to fully leverage transfer learning from pre-trained, open domain, causal language models such as GPT-2. SimpleTOD improves over the prior state-of-the-art in joint goal accuracy for dialogue state tracking, and our analysis reveals robustness to noisy annotations in this setting. SimpleTOD also improves the main metrics used to evaluate action decisions and response generation in an end-to-end…

GeDi: Generative Discriminator Guided Sequence Generation

Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann +4 more

cs.CL · September 2020 · cited 545× · arXiv ↗ · more like this ·

While large-scale language models (LMs) are able to imitate the distribution of natural language well enough to generate realistic text, it is difficult to control which regions of the distribution they generate. This is especially problematic because datasets used for training large LMs usually contain significant toxicity, hate, bias, and negativity. We propose GeDi as an efficient method for using smaller LMs as generative discriminators to guide generation from large LMs to make them safer and more controllable. GeDi guides generation at each step by computing classification probabilities for all possible next tokens via Bayes rule by normalizing over two class-conditional distributions; one conditioned on the desired attribute, or control code, and another conditioned on the undesired attribute, or anti control code. We find that GeDi gives stronger controllability than the state…

Neural Text Summarization: A Critical Evaluation

Wojciech Kryściński, Nitish Shirish Keskar, Bryan McCann +2 more

cs.CL · 23 Aug 2019 · cited 411× · arXiv ↗ · more like this ·

Text summarization aims at compressing long documents into a shorter form that conveys the most important parts of the original document. Despite increased interest in the community and notable research effort, progress on benchmark datasets has stagnated. We critically evaluate key ingredients of the current research setup: datasets, evaluation metrics, and models, and highlight three primary shortcomings: 1) automatically collected datasets leave the task underconstrained and may contain noise detrimental to training and evaluation, 2) current evaluation protocol is weakly correlated with human judgment and does not account for important characteristics such as factual correctness, 3) models overfit to layout biases of current datasets and offer limited diversity in their outputs.

CTRLsum: Towards Generic Controllable Text Summarization

Junxian He, Wojciech Kryściński, Bryan McCann +2 more

cs.CL · 8 Dec 2020 · cited 165× · arXiv ↗ · more like this ·

Current summarization systems yield generic summaries that are disconnected from users' preferences and expectations. To address this limitation, we present CTRLsum, a novel framework for controllable summarization. Our approach enables users to control multiple aspects of generated summaries by interacting with the summarization system through textual input in the form of a set of keywords or descriptive prompts. Using a single unified model, CTRLsum is able to achieve a broad scope of summary manipulation at inference time without requiring additional human annotations or pre-defining a set of control aspects during training. We quantitatively demonstrate the effectiveness of our approach on three domains of summarization datasets and five control aspects: 1) entity-centric and 2) length-controllable summarization, 3) contribution summarization on scientific papers, 4) invention…

XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and Question Answering

Jasdeep Singh, Bryan McCann, Nitish Shirish Keskar +2 more

cs.CL · 27 May 2019 · cited 85× · arXiv ↗ · more like this ·

While natural language processing systems often focus on a single language, multilingual transfer learning has the potential to improve performance, especially for low-resource languages. We introduce XLDA, cross-lingual data augmentation, a method that replaces a segment of the input text with its translation in another language. XLDA enhances performance of all 14 tested languages of the cross-lingual natural language inference (XNLI) benchmark. With improvements of up to $4.8\%$, training with XLDA achieves state-of-the-art performance for Greek, Turkish, and Urdu. XLDA is in contrast to, and performs markedly better than, a more naive approach that aggregates examples in various languages in a way that each example is solely in one language. On the SQuAD question answering task, we see that XLDA provides a $1.0\%$ performance increase on the English evaluation set. Comprehensive…

Media, Culture, and the Modern African American Freedom Struggle

Bryan McCann, Brian Ward

Sociology and Political Science · The Arkansas Historical Quarterly · 2002 · cited 84× · DOI ↗ · more like this ·

Stimulating and insightful, these essays on the relationship among the media, popular culture, and the postwar African American freedom struggle offer new perspectives on the nature of the Civil Rights Movement and its legacies. At the same time, they suggest how much the struggle itself shaped important trends in American culture and mass media in the 1950s and 1960s. Bringing together a range of voices seldom heard together, this book challenges readers to reconsider the ways in which a simplistic master narrative of the Movement has come to dominate popular, and even some scholarly, understandings of the meaning of the freedom struggle.

BERT is Not an Interlingua and the Bias of Tokenization

Jasdeep Singh, Bryan McCann, Richard Socher +1 more

Artificial Intelligence · 2019 · cited 70× · DOI ↗ · more like this ·

Multilingual transfer learning can benefit both high-and low-resource languages, but the source of these improvements is not well understood. Cananical Correlation Analysis (CCA) of the internal representations of a pretrained, multilingual BERT model reveals that the model partitions representations for each language rather than using a common, shared, interlingual space. This effect is magnified at deeper layers, suggesting that the model does not progressively abstract semantic content while disregarding languages. Hierarchical clustering based on the CCA similarity scores between languages reveals a tree structure that mirrors the phylogenetic trees hand-designed by linguists. The subword tokenization employed by BERT provides a stronger bias towards such structure than character-and wordlevel tokenizations. We release a subset of the XNLI dataset translated into an additional 14…

Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigation

Tianlu Wang, Xi Victoria Lin, Nazneen Fatema Rajani +3 more

cs.CL · 3 May 2020 · cited 68× · arXiv ↗ · more like this ·

Word embeddings derived from human-generated corpora inherit strong gender bias which can be further amplified by downstream models. Some commonly adopted debiasing approaches, including the seminal Hard Debias algorithm, apply post-processing procedures that project pre-trained word embeddings into a subspace orthogonal to an inferred gender subspace. We discover that semantic-agnostic corpus regularities such as word frequency captured by the word embeddings negatively impact the performance of these algorithms. We propose a simple but effective technique, Double Hard Debias, which purifies the word embeddings against such corpus regularities prior to inferring and removing the gender subspace. Experiments on three bias mitigation benchmarks show that our approach preserves the distributional semantics of the pre-trained word embeddings while reducing gender bias to a significantly…

Revisiting Activation Regularization for Language RNNs

Stephen Merity, Bryan McCann, Richard Socher

cs.CL · 3 Aug 2017 · cited 45× · arXiv ↗ · more like this ·

Recurrent neural networks (RNNs) serve as a fundamental building block for many sequence tasks across natural language processing. Recent research has focused on recurrent dropout techniques or custom RNN cells in order to improve performance. Both of these can require substantial modifications to the machine learning model or to the underlying RNN configurations. We revisit traditional regularization techniques, specifically L2 regularization on RNN activations and slowness regularization over successive hidden states, to improve the performance of RNNs on the task of language modeling. Both of these techniques require minimal modification to existing RNN architectures and result in performance improvements comparable or superior to more complicated regularization techniques or custom cell architectures. These regularization techniques can be used without any modification on optimized…

Turning Silence into Speech and Action: Prison Activism and the Pedagogy of Empowered Citizenship

Stephen John Hartnett, Jennifer K. Wood, Bryan McCann

Sociology and Political Science · Communication and Critical/Cultural Studies · 24 Sep 2011 · cited 36× · DOI ↗ · more like this ·

Based on almost 50 years of combined experience as prison activists and prison teachers, the authors offer three case studies of prison activism and pedagogy in action. The first case study, by Hartnett, details the “artistry of agency” as enacted in poetry workshops in prison and in public poetry events, thus illustrating artistic communication. The second, by Wood, examines how friendship becomes political in the epistolary communication between free and imprisoned correspondents, thus addressing interpersonal communication. The third, by McCann, addresses web-based communication as a tool for advocacy for condemned prisoner/activists on Texas’s death row, and hence political communication. Taken as a whole, the three case studies celebrate different communication strategies as avenues of enlightenment and empowerment while offering powerful arguments for abolishing the…

The Political Evolution of Rio de Janeiro's Favelas: Recent Works

Bryan McCann

Political Science and International Relations · Latin American Research Review · 2006 · cited 31× · DOI ↗ · more like this ·

The Political Evolution of Rio de Janeiro's Favelas Recent Works Reviewed by Bryan McCann Georgetown University A Favela Fala. Edited by Dulce Chaves Pandolfi and Mário Grynszpan. (Rio de Janeiro: Editora Fundação Getúlio Vargas, 2003. Pp. 364.) Integração Perversa: Pobreza E Tráfico De Drogas. By Alba Zaluar. (Rio de Janeiro: Editora Fundação Getúlio Vargas, 2004. Pp. 440.) Lucia: Testimonies of a Brazilian Drug Dealer's Woman. By Robert Gay. (Philadelphia: Temple University Press, 2005. Pp. 240. $59.50 cloth, $21.95 paper.) A Utopia Da Comunidade: Rio Das Pedras, Uma Favela Carioca. Edited by Marcelo Baumann Burgos. (Rio de Janeiro: Editora PUC-Rio. Pp. 232.) You've probably heard the one about Eskimo demography: how many Eskimos in the typical igloo? Five—a mother, father, two kids, and an anthropologist. The same joke might be made about Rio de Janeiro's favelas, but it would vastly…

Recalling Persky: White Rage and Intimate Publicity After Brock Turner

Ashley Noel Mack, Bryan McCann

Gender Studies · Journal of Communication Inquiry · 8 Aug 2019 · cited 30× · DOI ↗ · more like this ·

By attending to the successful campaign to recall and unseat Judge Aaron Persky in retaliation for his sentencing decision in the sexual assault trial of Stanford University athlete Brock Turner, we argue that the campaign constituted a White intimate public that directed White rage toward the judge for his perceived betrayals of a White paternal social contract. Because the campaign posited Persky’s willingness to inflict carceral violence upon Turner as the measure of his fidelity to the state’s obligations with regard to sexual violence and equality under the law, it legitimized carceral solutions that disproportionately harm communities of color. Such a centering of carceral solutions betrayed the public’s affective divestment from the needs of marginalized communities.

Unifying Question Answering, Text Classification, and Regression via Span Extraction

Nitish Shirish Keskar, Bryan McCann, Caiming Xiong +1 more

cs.CL · April 2019 · cited 21× · arXiv ↗ · more like this ·

Even as pre-trained language encoders such as BERT are shared across many tasks, the output layers of question answering, text classification, and regression models are significantly different. Span decoders are frequently used for question answering, fixed-class, classification layers for text classification, and similarity-scoring layers for regression tasks, We show that this distinction is not necessary and that all three can be unified as span extraction. A unified, span-extraction approach leads to superior or comparable performance in supplementary supervised pre-trained, low-data, and multi-task learning experiments on several question answering, text classification, and regression benchmarks.

Char2Subword: Extending the Subword Embedding Space Using Robust Character Compositionality

Gustavo Aguilar, Bryan McCann, Tong Niu +3 more

cs.CL · October 2020 · cited 17× · arXiv ↗ · more like this ·

Byte-pair encoding (BPE) is a ubiquitous algorithm in the subword tokenization process of language models as it provides multiple benefits. However, this process is solely based on pre-training data statistics, making it hard for the tokenizer to handle infrequent spellings. On the other hand, though robust to misspellings, pure character-level models often lead to unreasonably long sequences and make it harder for the model to learn meaningful words. To alleviate these challenges, we propose a character-based subword module (char2subword) that learns the subword embedding table in pre-trained models like BERT. Our char2subword module builds representations from characters out of the subword vocabulary, and it can be used as a drop-in replacement of the subword embedding table. The module is robust to character-level alterations such as misspellings, word inflection, casing, and…

Carlos Lacerda: The Rise and Fall of a Middle-Class Populist in 1950s Brazil

Bryan McCann

Sociology and Political Science · Hispanic American Historical Review · November 2003 · cited 14× · DOI ↗ · more like this ·

President Getúlio Vargas’s suicide on August 24, 1954, has long been recognized as the pivotal moment of Brazil’s Populist Republic—the tumultuous two decades between the fall of the Estado Novo dictatorship in 1945 and the military coup in 1964. His suicide—clearly as much grand political gesture as personal tragedy—passed almost immediately from current event to popular myth, interpreted in various ways according to fundamentally opposed visions of Brazil and its future. Vargas’s supporters saw the suicide as the martyrdom of a prince, and they saw the famous cartatestamento—Vargas’s disputed suicide note, typed several days in advance of the act—as a call to arms, urging the vigorous defense of Vargas’s legacy. Opponents, on the other hand, saw this as the last, cowardly act of a despicable traitor, a curse guaranteed to further muddy the waters of Brazilian politics for decades to…

“Strictly an act of street violence”: intimate publicity and affective divestment in the New Orleans Mother’s Day shooting

Ashley Noel Mack, Bryan McCann

Sociology and Political Science · Communication and Critical/Cultural Studies · 4 Sep 2017 · cited 10× · DOI ↗ · more like this ·

On May 12, 2013, two men opened fire on a Mother’s Day second line parade in New Orleans’s Seventh Ward. This essay attends to discourses from public officials and news media following the shooting that reified what we describe as affective divestment from the suffering of historically marginalized bodies and communities. Specifically, public discourse characterized the shooting as an episode of “street violence” that did not warrant sustained national attention. Affective divestment is the consequence of rhetorical maneuvers that signal a pushing away from certain bodies by intimate publics—an estrangement at the symbolic level that has naturalized or rationalized the neglect of certain forms of suffering. Such estrangement, we argue, is a function of neoliberal logics that devalue and, at times, necessitate the suffering of disposable populations.

African Rhythms in Brazilian Popular Music: Tango Brasileiro, Maxixe and Choro

Bryan McCann, Richard Louis Miller

Sociology and Political Science · Luso-Brazilian Review · June 2011 · cited 10× · DOI ↗ · more like this ·

As diferenças entre o maxixe e o tango brasileiro se definem mais claramente nos padrões de acompanhamento do violão. Este estudo define esses padrões e mostra a importância de entender as sutilezas das heranças Africana e Européia na música brasileira. Uma análise da evolução desses padrões de acompanhamento ilumina o desenvolvimento da música brasileira entre o fim do século XIX e os meados do século XX. A compreensão desses padrões influencia a interpretação das obras de Villa-Lobos e outros compositores brasileiros, e facilita uma maior apreciação da musica popular e erudita brasileira.

Joint Energy-based Model Training for Better Calibrated Natural Language Understanding Models

Tianxing He, Bryan McCann, Caiming Xiong +1 more

cs.CL · January 2021 · cited 8× · arXiv ↗ · more like this ·

In this work, we explore joint energy-based model (EBM) training during the finetuning of pretrained text encoders (e.g., Roberta) for natural language understanding (NLU) tasks. Our experiments show that EBM training can help the model reach a better calibration that is competitive to strong baselines, with little or no loss in accuracy. We discuss three variants of energy functions (namely scalar, hidden, and sharp-hidden) that can be defined on top of a text encoder, and compare them in experiments. Due to the discreteness of text data, we adopt noise contrastive estimation (NCE) to train the energy-based model. To make NCE training more effective, we train an auto-regressive noise model with the masked language model (MLM) objective.

Brazil's New Right

Bryan McCann

Political Science and International Relations · Dissent · 2018 · cited 6× · DOI ↗ · more like this ·

It seemed as if the Brazilian right had largely disappeared, as business elites prospered under two decades of a center-left administration, and cultural pluralism flourished. But in just the past few years, a new right has rapidly surged. A coalition of neo-liberals, evangelicals, and military government apologists, they are riven by contradictions but nevertheless committed to flexing their cultural and electoral muscle. In the aftermath President Dilma Roussef's 2016 impeachment and Brazil's descent into political irresolution, they threaten to become the decisive force in the October elections.