arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Adam Lopez

79 papers · 2005–2025

2005: 42006: 12007: 22008: 12009: 42010: 12011: 42012: 22013: 52014: 22015: 42016: 42017: 112018: 122019: 82020: 62021: 12022: 12023: 52024: 02025: 1
2005peak 12 in 20182025

Frequent co-authors

Papers

showing 1–25 of 79 RSS

cdec: A Decoder‚ Alignment‚ and Learning framework for finite−state and context−free translation models

Chris Dyer, Adam Lopez, Juri Ganitkevitch +6 more

Artificial Intelligence · Edinburgh Research Explorer (University of Edinburgh) · · cited 230× · OpenAlex ↗ · more like this ·

We present cdec, an open source frame-work for decoding, aligning with, and training a number of statistical machine translation models, including word-based models, phrase-based models, and models based on synchronous context-free gram-mars. Using a single unified internal representation for translation forests, the decoder strictly separates model-specific translation logic from general rescoring, pruning, and inference algorithms. From this unified representation, the decoder can extract not only the 1- or k-best transla-tions, but also alignments to a reference, or the quantities necessary to drive dis-criminative training using gradient-based or gradient-free optimization techniques. Its efficient C++ implementation means that memory use and runtime performance are significantly better than comparable decoders. 1

Pre-training on high-resource speech recognition improves low-resource speech-to-text translation

Sameer Bansal, Herman Kamper, Karen Livescu +2 more

cs.CL · · cited 221× · arXiv ↗ · more like this ·

We present a simple approach to improve direct speech-to-text translation (ST) when the source language is low-resource: we pre-train the model on a high-resource automatic speech recognition (ASR) task, and then fine-tune its parameters for ST. We demonstrate that our approach is effective by pre-training on 300 hours of English ASR data to improve Spanish-English ST from 10.8 to 20.2 BLEU when only 20 hours of Spanish-English ST training data are available. Through an ablation study, we find that the pre-trained encoder (acoustic model) accounts for most of the improvement, despite the fact that the shared language in these tasks is the target language text, not the source language audio. Applying this insight, we show that pre-training on ASR helps ST even when the ASR language differs from both source and target ST languages: pre-training on French ASR also improves Spanish-English…

Improved Speech-to-Text Translation with the Fisher and Callhome Spanish–English Speech Translation Corpus

Matt Post, Gaurav Kumar, Adam Lopez +3 more

Artificial Intelligence · Edinburgh Research Explorer · · cited 110× · OpenAlex ↗ · more like this ·

Research into the translation of the output of automatic speech recognition (ASR) systems is hindered by the dearth of datasets developed for that explicit purpose. For Spanish-English translation, in particular, most parallel data available exists only in vastly different domains and registers. In order to support research on cross-lingual speech applications, we introduce the Fisher and Callhome Spanish-English Speech Translation Corpus, supplementing existing LDC audio and transcripts with (a) ASR1-best, lattice, and oracle output produced by the Kaldi recognition system and (b) English translations obtained on Amazon’s Mechanical Turk. The result is a four-way parallel dataset of Spanish audio, transcriptions, ASR lattices, and English translations of approximately 38 hours of speech, with defined training, development, and<br/>held-out test sets.<br/><br/>We conduct baseline…

From Characters to Words to in Between: Do We Capture Morphology?

Clara Vania, Adam Lopez

cs.CL · · cited 104× · arXiv ↗ · more like this ·

Words can be represented by composing the representations of subword units such as word segments, characters, and/or character n-grams. While such representations are effective and may capture the morphological regularities of words, they have not been systematically compared, and it is not understood how they interact with different morphological typologies. On a language modeling task, we present experiments that systematically vary (1) the basic unit of representation, (2) the composition of these representations, and (3) the morphological typology of the language modeled. Our results extend previous findings that character representations are effective across typologies, and we find that a previously unstudied combination of character trigram representations composed with bi-LSTMs outperforms most others. But we also find room for improvement: none of the character-level models…

Dirt cheap web-scale parallel text from the Common Crawl

Jason Smith, Hervé Saint-Amand, Magdalena Plamadă +3 more

Artificial Intelligence · Zurich Open Repository and Archive (University of Zurich) · · cited 104× · DOI ↗ · more like this ·

Parallel text is the fuel that drives modern machine translation systems. The Web is a comprehensive source of preexisting parallel text, but crawling the entire web is impossible for all but the largest companies. We bring web-scale parallel text to the masses by mining the Common Crawl, a public Web crawl hosted on Amazon 19s Elastic Cloud. Starting from nothing more than a set of common two-letter language codes, our open-source extension of the STRAND algorithm mined 32 terabytes of the crawl in just under a day, at a cost of about \$500. Our large-scale experiment uncovers large amounts of parallel text in dozens of language pairs across a variety of domains and genres, some previously unavailable in curated datasets. Even with minimal cleaning and filtering, the resulting data boosts translation performance across the board for five different language pairs in the news domain, and…

Neural Networks For Negation Scope Detection

Federico Fancellu, Adam Lopez, Bonnie Webber

Artificial Intelligence · Edinburgh Research Explorer · · cited 96× · DOI ↗ · more like this ·

Automatic negation scope detection is a task that has been tackled using different classifiers and heuristics. Most systems are however 1) highly-engineered, 2) English-specific, and 3) only tested on the same genre they were trained on. We start by addressing 1) and 2) using a neural network architecture. Results obtained on data from the *SEM2012 shared task on negation scope detection show that even a simple feed-forward neural network using word-embedding features alone, performs on par with earlier classifiers, with a bi-directional LSTM outperforming all of them. We then address 3) by means of a specially-designed synthetic test set; in doing so, we explore the problem of detecting the negation scope more in depth and show that performance suffers from genre effects and differs with the type of negation considered.

Hierarchical Phrase-Based Translation with Suffix Arrays

Adam Lopez

Artificial Intelligence · Edinburgh Research Explorer (University of Edinburgh) · · cited 87× · OpenAlex ↗ · more like this ·

A major engineering challenge in statistical machine translation systems is the efficient representation of extremely large translation rulesets. In phrase-based models, this prob-lem can be addressed by storing the training data in memory and using a suffix array as an efficient index to quickly lookup and ex-tract rules on the fly. Hierarchical phrase-based translation introduces the added wrin-kle of source phrases with gaps. Lookup algorithms used for contiguous phrases no longer apply and the best approximate pat-tern matching algorithms are much too slow, taking several minutes per sentence. We describe new lookup algorithms for hierar-chical phrase-based translation that reduce the empirical computation time by nearly two orders of magnitude, making on-the-fly lookup feasible for source phrases with gaps. 1

Towards speech-to-text translation without speech recognition

Sameer Bansal, Herman Kamper, Adam Lopez +1 more

cs.CL · · cited 73× · arXiv ↗ · more like this ·

We explore the problem of translating speech to text in low-resource scenarios where neither automatic speech recognition (ASR) nor machine translation (MT) are available, but we have training data in the form of audio paired with text translations. We present the first system for this problem applied to a realistic multi-speaker dataset, the CALLHOME Spanish-English speech translation corpus. Our approach uses unsupervised term discovery (UTD) to cluster repeated patterns in the audio, creating a pseudotext, which we pair with translations to create a parallel text and train a simple bag-of-words MT model. We identify the challenges faced by the system, finding that the difficulty of cross-speaker UTD results in low recall, but that our system is still able to correctly translate some content words in test data.

Word-Based Alignment, Phrase-Based Translation: What's the Link?

Adam Lopez, Philip Resnik

Artificial Intelligence · Edinburgh Research Explorer (University of Edinburgh) · · cited 55× · OpenAlex ↗ · more like this ·

State-of-the-art statistical machine translation is based on alignments between phrases – sequences of words in the source and target sentences. The learning step in these systems often relies on alignments between words. It is often assumed that the quality of this word alignment is critical for translation. However, recent results suggest that the relationship between alignment quality and translation quality is weaker than previously thought. We investigate this question directly, comparing the impact of highquality alignments with a carefully constructed set of degraded alignments. In order to tease apart various interactions, we report experiments investigating the impact of alignments on different aspects of the system. Our results confirm a weak correlation, but they also illustrate that more data and better feature engineering may be more beneficial than better alignment. 1

A systematic comparison of methods for low-resource dependency parsing on genuinely low-resource languages

Clara Vania, Yova Kementchedjhieva, Anders Søgaard +1 more

cs.CL · · cited 52× · arXiv ↗ · more like this ·

Parsers are available for only a handful of the world's languages, since they require lots of training data. How far can we get with just a small amount of training data? We systematically compare a set of simple strategies for improving low-resource parsers: data augmentation, which has not been tested before; cross-lingual training; and transliteration. Experimenting on three typologically diverse low-resource languages---North Sámi, Galician, and Kazah---We find that (1) when only the low-resource treebank is available, data augmentation is very helpful; (2) when a related high-resource treebank is available, cross-lingual training is helpful and complements data augmentation; and (3) when the high-resource treebank uses a different writing system, transliteration into a shared orthographic spaces is also very helpful.

A Unified Framework for Phrase-Based, Hierarchical, and Syntax-Based Statistical Machine Translation

Hieu Hoang, Philipp Koehn, Adam Lopez

Artificial Intelligence · IWSLT · · cited 52× · OpenAlex ↗ · more like this ·

Despite many differences between phrase-based, hier-archical, and syntax-based translation models, their train-ing and testing pipelines are strikingly similar. Draw-ing on this fact, we extend the Moses toolkit to imple-ment hierarchical and syntactic models, making it the first open source toolkit with end-to-end support for all three of these popular models in a single package. This extension substantially lowers the barrier to entry for machine translation research across multiple models. 1.

A Comparison of Loopy Belief Propagation and Dual Decomposition for Integrated CCG Supertagging and Parsing

Michael Auli, Adam Lopez

Artificial Intelligence · Edinburgh Research Explorer (University of Edinburgh) · · cited 50× · OpenAlex ↗ · more like this ·

Via an oracle experiment, we show that the upper bound on accuracy of a CCG parser is significantly lowered when its search space is pruned using a supertagger, though the supertagger also prunes many bad parses. Inspired by this analysis, we design a single model with both supertagging and parsing features, rather than separating them into distinct models chained together in a pipeline. To overcome the resulting increase in complexity, we experiment with both belief propagation and dual decomposition approaches to inference, the first empirical comparison of these algorithms that we are aware of on a structured natural language processing problem. On CCGbank we achieve a labelled dependency F-measure of 88.8 % on gold POS tags, and 86.7 % on automatic part-of-speeoch tags, the best reported results for this task. 1

Tera-scale translation models via pattern matching

Adam Lopez

Artificial Intelligence · Edinburgh Research Explorer · · cited 50× · DOI ↗ · more like this ·

Translation model size is growing at a pace that outstrips improvements in computing power, and this hinders research on many interesting models. We show how an algorithmic scaling technique can be used to easily handle very large models. Using this technique, we explore several large model variants and show an improvement 1.4 BLEU on the NIST 2006 Chinese-English task. This opens the door for work on a variety of models that are much less constrained by computational limitations.

The Hiero machine translation system

David Chiang, Adam Lopez, Nitin Madnani +3 more

Artificial Intelligence · · cited 47× · DOI ↗ · more like this ·

Hierarchical organization is a well known property of language, and yet the notion of hierarchical structure has been largely absent from the best performing machine translation systems in recent community-wide evaluations. In this paper, we discuss a new hierarchical phrase-based statistical machine translation system (Chiang, 2005), presenting recent extensions to the original proposal, new evaluation results in a community-wide evaluation, and a novel technique for fine-grained comparative analysis of MT systems.

Inflecting when there's no majority: Limitations of encoder-decoder neural networks as cognitive models for German plurals

Kate McCurdy, Sharon Goldwater, Adam Lopez

cs.CL · · cited 40× · arXiv ↗ · more like this ·

Can artificial neural networks learn to represent inflectional morphology and generalize to new words as human speakers do? Kirov and Cotterell (2018) argue that the answer is yes: modern Encoder-Decoder (ED) architectures learn human-like behavior when inflecting English verbs, such as extending the regular past tense form -(e)d to novel words. However, their work does not address the criticism raised by Marcus et al. (1995): that neural models may learn to extend not the regular, but the most frequent class -- and thus fail on tasks like German number inflection, where infrequent suffixes like -s can still be productively generalized. To investigate this question, we first collect a new dataset from German speakers (production and ratings of plural forms for novel nouns) that is designed to avoid sources of information unavailable to the ED model. The speaker data show high…

Translation as weighted deduction

Adam Lopez

Artificial Intelligence · Edinburgh Research Explorer · · cited 40× · DOI ↗ · more like this ·

We present a unified view of many translation algorithms that synthesizes work on deductive parsing, semiring parsing, and efficient approximate search algorithms. This gives rise to clean analyses and compact descriptions that can serve as the basis for modular implementations. We illustrate this with several examples, showing how to build search spaces for several disparate phrase-based search strategies, integrate non-local features, and devise novel models. Although the framework is drawn from parsing and applied to translation, it is applicable to many dynamic programming problems arising in natural language processing and other areas.

Detecting negation scope is easy, except when it isn't

Federico Fancellu, Adam Lopez, Bonnie Webber +1 more

Artificial Intelligence · Edinburgh Research Explorer (University of Edinburgh) · · cited 33× · DOI ↗ · more like this ·

Several corpora have been annotated with negation scope-the set of words whose meaning is negated by a cue like the word "not"-leading to the development of classifiers that detect negation scope with high accuracy. We show that for nearly all of these corpora, this high accuracy can be attributed to a single fact: they frequently annotate negation scope as a single span of text delimited by punctuation. For negation scopes not of this form, detection accuracy is low and undersampling the easy training examples does not substantially improve accuracy. We demonstrate that this is partly an artifact of annotation guidelines, and we argue that future negation scope annotation efforts should focus on these more difficult cases.

What do character-level models learn about morphology? The case of dependency parsing

Clara Vania, Andreas Grivas, Adam Lopez

cs.CL · · cited 28× · arXiv ↗ · more like this ·

When parsing morphologically-rich languages with neural models, it is beneficial to model input at the character level, and it has been claimed that this is because character-level models learn morphology. We test these claims by comparing character-level models to an oracle with access to explicit morphological analysis on twelve languages with varying morphological typologies. Our results highlight many strengths of character-level models, but also show that they are poor at disambiguating some words, particularly in the face of case syncretism. We then demonstrate that explicitly modeling morphological case improves our best model, showing that character-level models can benefit from targeted forms of explicit morphological modeling.

Understanding Learning Dynamics Of Language Models with SVCCA

Naomi Saphra, Adam Lopez

cs.CL · · cited 27× · arXiv ↗ · more like this ·

Research has shown that neural models implicitly encode linguistic features, but there has been no research showing \emph{how} these encodings arise as the models are trained. We present the first study on the learning dynamics of neural language models, using a simple and flexible analysis method called Singular Vector Canonical Correlation Analysis (SVCCA), which enables us to compare learned representations across time and across models, without the need to evaluate directly on annotated data. We probe the evolution of syntactic, semantic, and topic representations and find that part-of-speech is learned earlier than topic; that recurrent layers become more similar to those of a tagger during training; and embedding layers less similar. Our results and methods could inform better learning algorithms for NLP models, possibly to incorporate linguistic information more effectively.

Training a Log-Linear Parser with Loss Functions via Softmax-Margin

Michael Auli, Adam Lopez

Artificial Intelligence · Edinburgh Research Explorer · · cited 27× · OpenAlex ↗ · more like this ·

Log-linear parsing models are often trained by optimizing likelihood, but we would prefer to optimise for a task-specific metric like F-measure. Softmax-margin is a convex objective for such models that minimises a bound on expected risk for a given loss function, but its naïve application requires the loss to decompose over the predicted structure, which is not true of F-measure. We use softmaxmargin to optimise a log-linear CCG parser for a variety of loss functions, and demonstrate a novel dynamic programming algorithm that enables us to use it with F-measure, leading to substantial gains in accuracy on CCG-Bank. When we embed our loss-trained parser into a larger model that includes supertagging features incorporated via belief propagation, we obtain further improvements and achieve a labelled/unlabelled dependency F-measure of 89.3%/94.0 % on gold part-of-speech tags, and…

EggCountAI: a convolutional neural network-based software for counting of Aedes aegypti mosquito eggs

Nouman Javed, Adam J. López‐Denman, Prasad N. Paradkar +1 more

Computer Vision and Pattern Recognition · Parasites & Vectors · · cited 26× · DOI ↗ · more like this ·

BACKGROUND: Mosquito-borne diseases exert a huge impact on both animal and human populations, posing substantial health risks. The behavioural and fitness traits of mosquitoes, such as locomotion and fecundity, are crucial factors that influence the spread of diseases. In existing egg-counting tools, each image requires separate processing with adjustments to various parameters such as intensity threshold and egg area size. Furthermore, accuracy decreases significantly when dealing with clustered or overlapping eggs. To overcome these issues, we have developed EggCountAI, a Mask Region-based Convolutional Neural Network (RCNN)-based free automatic egg-counting tool for Aedes aegypti mosquitoes. METHODS: The study design involves developing EggCountAI for counting mosquito eggs and comparing its performance with two commonly employed tools-ICount and MECVision-using 10 microscopic and 10…

Joshua 3.0: Syntax-based Machine Translation with the Thrax Grammar Extractor

Jonathan Weese, Juri Ganitkevitch, Chris Callison-Burch +2 more

Artificial Intelligence · Edinburgh Research Explorer (University of Edinburgh) · · cited 26× · OpenAlex ↗ · more like this ·

We present progress on Joshua, an opensource decoder for hierarchical and syntaxbased machine translation. The main focus is describing Thrax, a flexible, open source synchronous context-free grammar extractor. Thrax extracts both hierarchical (Chiang, 2007) and syntax-augmented machine translation (Zollmann and Venugopal, 2006) grammars. It is built on Apache Hadoop for efficient distributed performance, and can easily be extended with support for new grammars, feature functions, and output formats. 1

A systematic analysis of translation model search spaces

Michael Auli, Adam Lopez, Hieu Hoang +1 more

Artificial Intelligence · Edinburgh Research Explorer · · cited 26× · DOI ↗ · more like this ·

Translation systems are complex, and most metrics do little to pinpoint causes of error or isolate system differences. We use a simple technique to discover induction errors, which occur when good translations are absent from model search spaces. Our results show that a common pruning heuristic drastically increases induction error, and also strongly suggest that the search spaces of phrase-based and hierarchical phrase-based models are highly overlapping despite the well known structural differences.

Monte carlo inference and maximization for phrase-based translation

Abhishek Arun, Chris Dyer, Barry Haddow +3 more

Artificial Intelligence · Oxford University Research Archive (ORA) (University of Oxford) · · cited 25× · DOI ↗ · more like this ·

Recent advances in statistical machine translation have used beam search for approximate NP-complete inference within probabilistic translation models. We present an alternative approach of sampling from the posterior distribution defined by a translation model. We define a novel Gibbs sampler for sampling translations given a source sentence and show that it effectively explores this posterior distribution. In doing so we overcome the limitations of heuristic beam search and obtain theoretically sound solutions to inference problems such as finding the maximum probability translation and minimum expected risk training and decoding.

Semantic Graph Parsing with Recurrent Neural Network DAG Grammars

Federico Fancellu, Sorcha Gilroy, Adam Lopez +1 more

cs.CL · · cited 24× · arXiv ↗ · more like this ·

Semantic parses are directed acyclic graphs (DAGs), so semantic parsing should be modeled as graph prediction. But predicting graphs presents difficult technical challenges, so it is simpler and more common to predict the linearized graphs found in semantic parsing datasets using well-understood sequence models. The cost of this simplicity is that the predicted strings may not be well-formed graphs. We present recurrent neural network DAG grammars, a graph-aware sequence model that ensures only well-formed graphs while sidestepping many difficulties in graph prediction. We test our model on the Parallel Meaning Bank---a multilingual semantic graphbank. Our approach yields competitive results in English and establishes the first results for German, Italian and Dutch.