arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Keith Hall

87 papers · 1910–2024

1965: 01966: 01967: 01968: 01969: 01970: 01971: 01972: 01973: 11974: 11975: 01976: 01977: 01978: 01979: 01980: 01981: 01982: 01983: 01984: 01985: 01986: 01987: 01988: 01989: 01990: 01991: 01992: 11993: 01994: 01995: 01996: 01997: 01998: 41999: 02000: 22001: 02002: 02003: 02004: 32005: 22006: 22007: 52008: 12009: 72010: 82011: 92012: 22013: 62014: 42015: 72016: 32017: 22018: 02019: 22020: 42021: 22022: 42023: 12024: 2
1965peak 9 in 20112024

Frequent co-authors

Papers

showing 1–25 of 87 RSS

Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models

Jianmo Ni, Gustavo Hernández Ábrego, Noah Constant +4 more

cs.CL · · cited 890× · arXiv ↗ · more like this ·

We provide the first exploration of sentence embeddings from text-to-text transformers (T5). Sentence embeddings are broadly useful for language processing tasks. While T5 achieves impressive performance on language tasks cast as sequence-to-sequence mapping problems, it is unclear how to produce sentence embeddings from encoder-decoder models. We investigate three methods for extracting T5 sentence embeddings: two utilize only the T5 encoder and one uses the full T5 encoder-decoder model. To support our investigation, we establish a new sentence representation transfer benchmark, SentGLUE, which extends the SentEval toolkit to nine tasks from the GLUE benchmark. Our encoder-only models outperforms Sentence-BERT and SimCSE sentence embeddings on both SentEval and SentGLUE transfer tasks, including semantic textual similarity (STS). Scaling up T5 from millions to billions of parameters…

A study on similarity and relatedness using distributional and WordNet-based approaches

Eneko Agirre, Enrique Alfonseca, Keith Hall +3 more

Artificial Intelligence · · cited 883× · DOI ↗ · more like this ·

This paper presents and compares WordNet-based and distributional similarity approaches. The strengths and weaknesses of each approach regarding similarity and relatedness tasks are discussed, and a combination is presented. Each of our methods independently provide the best results in their class on the RG and WordSim353 datasets, and a supervised combination of them yields the best published results on all datasets. Finally, we pioneer cross-lingual similarity, showing that our methods are easily adapted for a cross-lingual task with minor losses.

Large Dual Encoders Are Generalizable Retrievers

Jianmo Ni, Chen Qu, Jing Lu +8 more

cs.IR · · cited 678× · arXiv ↗ · more like this ·

It has been shown that dual encoders trained on one domain often fail to generalize to other domains for retrieval tasks. One widespread belief is that the bottleneck layer of a dual encoder, where the final score is simply a dot-product between a query vector and a passage vector, is too limited to make dual encoders an effective retrieval model for out-of-domain generalization. In this paper, we challenge this belief by scaling up the size of the dual encoder model {\em while keeping the bottleneck embedding size fixed.} With multi-stage training, surprisingly, scaling up the model size brings significant improvement on a variety of retrieval tasks, especially for out-of-domain generalization. Experimental results show that our dual encoders, \textbf{G}eneralizable \textbf{T}5-based dense \textbf{R}etrievers (GTR), outperform %ColBERT~\cite{khattab2020colbert} and existing sparse and…

Universal Dependency Annotation for Multilingual Parsing

Ryan McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage +9 more

Artificial Intelligence · Meeting of the Association for Computational Linguistics · · cited 443× · OpenAlex ↗ · more like this ·

We present a new collection of treebanks with homogeneous syntactic dependency annotation for six languages: German, English, Swedish, Spanish, French and Korean. To show the usefulness of such a resource, we present a case study of crosslingual transfer parsing with more reliable evaluation than has been possible before. This ‘universal ’ treebank is made freely available in order to facilitate research on multilingual dependency parsing. 1 1

The Political Economy of Administered Protection

Jürgen Finger, H. Keith Hall, Douglas Nelson

Political Science and International Relations · World Scientific Studies in International Economics · · cited 375× · DOI ↗ · more like this ·

Trade restrictions are hardly ever voted directly by Congress. Though the threat of direct congressional action is frequently present, pressure for protection is usually applied through the major instruments for the administrative regulation of imports: the antidumping and countervailing duty procedures and the escape clause mechanism. In this paper, we will examine these instruments for the administrative regulation of imports. The form of the paper is not novel -a theory of how these mechanisms work is developed, then operational implications of this theory are deduced and tested over the record of countervailing duty, antidumping, and escape clause cases decided from 1975 through 1979. This is the period over which the Trade Act of 1974 was in force. Because the nature of these mechanisms is relatively unfamiliar, a relatively large amount of institutional information is provided.'

Promptagator: Few-shot Dense Retrieval From 8 Examples

Zhuyun Dai, Vincent Y. Zhao, Ji Ma +7 more

cs.CL · · cited 370× · arXiv ↗ · more like this ·

Much recent research on information retrieval has focused on how to transfer from one task (typically with abundant supervised data) to various other tasks where supervision is limited, with the implicit assumption that it is possible to generalize from one task to all the rest. However, this overlooks the fact that there are many diverse and unique retrieval tasks, each targeting different search intents, queries, and search domains. In this paper, we suggest to work on Few-shot Dense Retrieval, a setting where each task comes with a short description and a few examples. To amplify the power of a few examples, we propose Prompt-base Query Generation for Retriever (Promptagator), which leverages large language models (LLM) as a few-shot query generator, and creates task-specific retrievers based on the generated data. Powered by LLM's generalization ability, Promptagator makes it…

Multi-Source Transfer of Delexicalized Dependency Parsers

Ryan McDonald, Slav Petrov, Keith Hall

Artificial Intelligence · Empirical Methods in Natural Language Processing · · cited 314× · OpenAlex ↗ · more like this ·

We present a simple method for transferring dependency parsers from source languages with labeled training data to target languages without labeled training data. We first demonstrate that delexicalized parsers can be directly transferred between languages, producing significantly higher accuracies than unsupervised parsers. We then use a constraint driven learning algorithm where constraints are drawn from parallel corpora to project the final parser. Unlike previous work on projecting syntactic resources, we show that simple methods for introducing multiple source languages can significantly improve the overall quality of the resulting parsers. The projected parsers from our system result in state-of-the-art performance when compared to previously studied unsupervised and projected parsing systems across eight different languages.

The first proposal for a permanent international criminal court

Christopher Keith Hall

Political Science and International Relations · International Review of the Red Cross · · cited 279× · DOI ↗ · more like this ·

On 15 June 1998, a diplomatic conference in Rome will open a five-week session to remedy one of the long-standing gaps in the implementation system for international humanitarian law by adopting a treaty to establish a permanent international criminal court. Although the current effort within the United Nations to set up a permanent court began half a century ago with a proposal in 1947 by Henri Donnedieu de Vabres, the French judge on the International Military Tribunal at Nuremberg, it is not widely known that the first serious such proposal appears to have been made more than a century and a quarter ago by Gustave Moynier, one of the founders, and longtime President, of the International Committee of the Red Cross. He wrestled with many of the same problems which will face the drafters of the statute at the 1998 diplomatic conference and the strengths and weaknesses of his proposal…

Distributed Training Strategies for the Structured Perceptron

Ryan McDonald, Keith Hall, Gideon Mann

Artificial Intelligence · North American Chapter of the Association for Computational Linguistics · · cited 265× · OpenAlex ↗ · more like this ·

Perceptron training is widely applied in the natural language processing community for learning complex structured models. Like all structured prediction learning frameworks, the structured perceptron can be costly to train as training complexity is proportional to inference, which is frequently non-linear in example sequence length. In this paper we investigate distributed training strategies for the structured perceptron as a means to reduce training times when computing clusters are available. We look at two strategies and provide convergence bounds for a particular mode of distributed structured perceptron training based on iterative parameter mixing (or averaging). We present experiments on two structured prediction problems – namedentity recognition and dependency parsing – to highlight the efficiency of this method. 1

SmallWorlds: Visualizing Social Recommendations

Brynjar Gretarsson, John O’Donovan, Svetlin Bostandjiev +2 more

Information Systems · Computer Graphics Forum · · cited 111× · DOI ↗ · more like this ·

Abstract We present SmallWorlds, a visual interactive graph‐based interface that allows users to specify, refine and build item‐preference profiles in a variety of domains. The interface facilitates expressions of taste through simple graph interactions and these preferences are used to compute personalized, fully transparent item recommendations for a target user. Predictions are based on a collaborative analysis of preference data from a user's direct peer group on a social network. We find that in addition to receiving transparent and accurate item recommendations, users also learn a wealth of information about the preferences of their peers through interaction with our visualization. Such information is not easily discoverable in traditional text based interfaces. A detailed analysis of our design choices for visual layout, interaction and prediction techniques is presented. Our…

Processing South Asian Languages Written in the Latin Script: the Dakshina Dataset

Brian Roark, Lawrence Wolf-Sonkin, Christo Kirov +4 more

cs.CL · · cited 99× · arXiv ↗ · more like this ·

This paper describes the Dakshina dataset, a new resource consisting of text in both the Latin and native scripts for 12 South Asian languages. The dataset includes, for each language: 1) native script Wikipedia text; 2) a romanization lexicon; and 3) full sentence parallel data in both a native script of the language and the basic Latin alphabet. We document the methods used for preparation and selection of the Wikipedia text in each language; collection of attested romanizations for sampled lexicons; and manual romanization of held-out sentences from the native script collections. We additionally provide baseline results on several tasks made possible by the dataset, including single word transliteration, full sentence transliteration, and language modeling of native script and romanized text. Keywords: romanization, transliteration, South Asian languages

Bringing contextual information to google speech recognition

Petar Aleksic, Mohammadreza Ghodsi, Assaf Michaely +5 more

Artificial Intelligence · · cited 96× · DOI ↗ · more like this ·

In automatic speech recognition on mobile devices, very often what a user says strongly depends on the particular context he or she is in. The n-grams relevant to the context are often not known in advance. The context can depend on, for example, particular dialog state, options presented to the user, conversation topic, location, etc. Speech recognition of sentences that include these n-grams can be challenging, as they are often not well represented in a language model (LM) or even include out-of-vocabulary (OOV) words. In this paper, we propose a solution for using contextual information to improve speech recognition accuracy. We utilize an on-the-fly rescoring mechanism to adjust the LM weights of a small set of n-grams relevant to the particular context during speech decoding. Our solution handles out of vocabulary words. It also addresses efficient combination of multiple sources…

The Third and Fourth Sessions of the UN Preparatory Committee on the Establishment of an International Criminal Court

Christopher Keith Hall

Political Science and International Relations · American Journal of International Law · · cited 96× · DOI ↗ · more like this ·

The Preparatory Committee on the Establishment of an International Criminal Court (Preparatory Committee or committee) held its third and fourth sessions from February 10 to 21, 1997, and August 4 to 15, 1997, bringing the world a step closer to the establishment of a permanent international criminal court (ICC). The United Nations General Assembly, on December 16, 1996, reaffirmed the mandate of the Preparatory Committee and decided that the committee would meet in four more sessions (the final two from December 1 to 12, 1997, and March 16 to April 3, 1998) “in order to complete the drafting of a widely acceptable consolidated text of a convention, to be submitted to a diplomatic conference.” It also decided that “a diplomatic conference of plenipotentiaries will be held in 1998, with a view to finalizing and adopting a convention on the establishment of an international criminal…

The First Five Sessions of the UN Preparatory Commission for the International Criminal Court

Christopher Keith Hall

Political Science and International Relations · American Journal of International Law · · cited 91× · DOI ↗ · more like this ·

On June 30, 2000, at the conclusion of its fifth session, the Preparatory Commission for the International Criminal Court (Preparatory Commission or commission) completed the first phase of its work with the adoption of draft Elements of Crimes and Rules of Procedure and Evidence (Rules). The first five sessions took place from February 16 to 26, July 26 to August 13, and November 29 to December 17,1999, and from March 13 to 31, and J u n e 12 to 30, 2000. On July 17, 1998, the United Nations Conference of Plenipotentiaries on the Establishment of the International Criminal Court (Diplomatic Conference) had adopted the Rome Statute of the International Criminal Court (Statute). At the same time, it decided “to take all possible measures to ensure the coming into operation of the International Criminal Court without undue delay and to make the necessary arrangements for the commencement…

Improved video categorization from text metadata and user comments

Katja Filippova, Keith Hall

Computer Vision and Pattern Recognition · · cited 72× · DOI ↗ · more like this ·

We consider the task of assigning categories (e.g., howto/cooking, sports/basketball, pet/dogs) to YouTube videos from video and text signals. We show that two complementary views on the data -- from the video and text perspectives -- complement each other and refine predictions. The contributions of the paper are threefold: (1) we show that a text-based classifier trained on imperfect predictions of the weakly supervised video content-based classifier is not redundant; (2) we demonstrate that a simple model which combines the predictions made by the two classifiers outperforms each of them taken independently; (3) we analyse such sources of text information as video title, description, user tags and viewers' comments and show that each of them provides valuable clues to the topic of the video.

The Fifth Session of the UN Preparatory Committee on the Establishment of an International Criminal Court

Christopher Keith Hall

Political Science and International Relations · American Journal of International Law · · cited 72× · DOI ↗ · more like this ·

The Preparatory Committee on the Establishment of an International Criminal Court (Preparatory Committee or committee) held its fifth session from December 1 to 12, 1997. The Preparatory Committee, chaired by Adriaan Bos (the Netherlands), conducted most of its work in five open working groups to consider (1) definitions of war crimes, (2) general principles of criminal law, (3) penalties, (4) procedural matters, and (5) state cooperation. Three of the working groups were assisted by small drafting groups focusing on particular articles; in the other two working groups, the chairs undertook the task of drafting revised texts in the light of the discussion. The following short summary discusses some of the major decisions adopted by the five working groups; the positions of some of the states, including the United States, that were active on these issues; and the plans of the Preparatory…

Composition-based on-the-fly rescoring for salient n-gram biasing

Keith Hall, Eunjoon Cho, Cyril Allauzen +7 more

Artificial Intelligence · · cited 56× · DOI ↗ · more like this ·

We introduce a technique for dynamically applying contextually-derived language models to a state-of-the-art speech recognition system. These generally small-footprint models can be seen as a generalization of cache-based models [1], whereby contextually salient n-grams are derived from relevant sources (not just user generated language) to produce a model intended for combination with the baseline language model. The derived models are applied during first-pass decoding as a form of on-the-fly composition between the decoder search graph and the set of weighted contextual n-grams. We present a construction algorithm which takes a trie representing the contextual n-grams and produces a weighted finite state automaton which is more compact than a standard n-gram machine. Finally, we present a set of empirical results on the recognition of spoken search queries where a contextual model…

The Sixth Session of the UN Preparatory Committee on the Establishment of an International Criminal Court

Christopher Keith Hall

Political Science and International Relations · American Journal of International Law · · cited 55× · DOI ↗ · more like this ·

The Preparatory Committee on the Establishment of an International Criminal Court (Preparatory Committee or committee) held its sixth and final session from March 16 to April 3, 1998, at the United Nations headquarters in New York. At this session it completed its work of preparing a consolidated text of a statute for a permanent international criminal court (ICC) for adoption at a diplomatic conference in Rome held from June 15 to July 17, 1998. The consolidated text is considerably longer than the draft statute submitted by the International Law Commission to the UN General Assembly in July 1994 (ILC draft statute) and contains many different options submitted by states, but it is still largely consistent with the basic structure of the ILC draft statute.

K-best Spanning Tree Parsing

Keith Hall

Artificial Intelligence · · cited 47× · OpenAlex ↗ · more like this ·

This paper introduces a Maximum Entropy dependency parser based on an efficient k-best Maximum Spanning Tree (MST) algo-rithm. Although recent work suggests that the edge-factored constraints of the MST al-gorithm significantly inhibit parsing accu-racy, we show that generating the 50-best parses according to an edge-factored model has an oracle performance well above the 1-best performance of the best dependency parsers. This motivates our parsing ap-proach, which is based on reranking the k-best parses generated by an edge-factored model. Oracle parse accuracy results are presented for the edge-factored model and 1-best results for the reranker on eight lan-guages (seven from CoNLL-X and English). 1

MapReduce/Bigtable for Distributed Optimization

Keith Hall, Scott Gilpin, Gideon Mann

Artificial Intelligence · · cited 46× · OpenAlex ↗ · more like this ·

With large data sets, it can be time consuming to run gradient based optimization, for example to minimize the log-likelihood for maximum entropy models. Distributed methods are therefore appealing and a number of distributed gradient optimization strategies have been proposed including: distributed gradient, asynchronous updates, and iterative parameter mixtures. In this paper, we evaluate these various strategies with regards to their accuracy and speed over MapReduce/Bigtable and discuss techniques and configurations needed for high performance. 1

Corrective modeling for non-projective dependency parsing

Keith Hall, Václav Novák

Artificial Intelligence · · cited 45× · DOI ↗ · more like this ·

We present a corrective model for recovering non-projective dependency structures from trees generated by state-of-the-art constituency-based parsers. The continuity constraint of these constituency-based parsers makes it impossible for them to posit non-projective dependency trees. Analysis of the types of dependency errors made by these parsers on a Czech corpus show that the correct governor is likely to be found within a local neighborhood of the governor proposed by the parser. Our model, based on a MaxEnt classifier, improves overall dependency accuracy by .7% (a 4.5% reduction in error) with over 50% accuracy for non-projective structures.

The Powers and Role of the Prosecutor of the International Criminal Court in the Global Fight against Impunity

Christopher Keith Hall

Political Science and International Relations · Leiden Journal of International Law · · cited 34× · DOI ↗ · more like this ·

On 16 June 2003, the first Prosecutor of the newly established International Criminal Court (Court), Luis Moreno Ocampo, was inaugurated. He faces enormous challenges ahead in the short term, including the need to increase the number of states ratifying and implementing the Rome Statute of the International Criminal Court and to demonstrate that criticisms of the Court and his powers made by the current administration of the United States of America in the course of its campaign to undermine the Court are unwarranted. This article describes the background to the establishment of a permanent independent Prosecutor within the Court, able to open, subject to extensive statutory and judicial constraints, investigations on the Prosecutor's own initiative. It then describes the statutory provisions establishing the post and defining the powers and duties of the Prosecutor. The article…

Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation

Ji Ma, Ivan Korotkov, Yinfei Yang +2 more

cs.IR · · cited 32× · arXiv ↗ · more like this ·

A major obstacle to the wide-spread adoption of neural retrieval models is that they require large supervised training sets to surpass traditional term-based techniques, which are constructed from raw corpora. In this paper, we propose an approach to zero-shot learning for passage retrieval that uses synthetic question generation to close this gap. The question generation system is trained on general domain data, but is applied to documents in the targeted domain. This allows us to create arbitrarily large, yet noisy, question-passage relevance pairs that are domain specific. Furthermore, when this is coupled with a simple hybrid term-neural model, first-stage retrieval performance can be improved further. Empirically, we show that this is an effective strategy for building neural passage retrieval models in the absence of large training corpora. Depending on the domain, this technique…

Reconstructing false start errors in spontaneous speech text

Erin Fitzgerald, Keith Hall, Frederick Jelinek

Artificial Intelligence · · cited 30× · DOI ↗ · more like this ·

This paper presents a conditional random field-based approach for identifying speaker-produced disfluencies (i.e. if and where they occur) in spontaneous speech transcripts. We emphasize false start regions, which are often missed in current disfluency identification approaches as they lack lexical or structural similarity to the speech immediately following. We find that combining lexical, syntactic, and language model-related features with the output of a state-of-the-art disfluency identification system improves overall word-level identification of these and other errors. Improvements are reinforced under a stricter evaluation metric requiring exact matches between cleaned sentences annotator-produced reconstructions, and altogether show promise for general reconstruction efforts.

Generalized syntactic and semantic models of query reformulation

Amaç Herdağdelen, Massimiliano Ciaramita, Daniel Mahler +4 more

Computer Networks and Communications · · cited 29× · DOI ↗ · more like this ·

We present a novel approach to query reformulation which combines syntactic and semantic information by means of generalized Levenshtein distance algorithms where the substitution operation costs are based on probabilistic term rewrite functions. We investigate unsupervised, compact and efficient models, and provide empirical evidence of their effectiveness. We further explore a generative model of query reformulation and supervised combination methods providing improved performance at variable computational costs. Among other desirable properties, our similarity measures incorporate information-theoretic interpretations of taxonomic relations such as specification and generalization.