Author

Rob Koeling

32 papers · 1996–2022

1996peak 5 in 20042022

Frequent co-authors

Papers

showing 1–25 of 32 RSS

Finding predominant word senses in untagged text

Diana McCarthy, Rob Koeling, Julie Weeds +1 more

Artificial Intelligence · Figshare · 2004 · cited 315× · DOI ↗ · more like this ·

In word sense disambiguation (WSD), the heuristic of choosing the most common sense is extremely powerful because the distribution of the senses of a word is often skewed. The problem with using the predominant, or first sense heuristic, aside from the fact that it does not take surrounding context into account, is that it assumes some quantity of hand-tagged data. Whilst there are a few hand-tagged corpora available for some languages, one would expect the frequency distribution of the senses of words, particularly topical words, to depend on the genre and domain of the text under consideration. We present work on the use of a thesaurus acquired from raw textual corpora and the WordNet similarity package to find predominant noun senses automatically. The acquired predominant senses give a precision of 64% on the nouns of the SENSEVAL-2 English all-words task. This is a very promising…

Unsupervised Acquisition of Predominant Word Senses

Diana McCarthy, Rob Koeling, Julie Weeds +1 more

Artificial Intelligence · Computational Linguistics · December 2007 · cited 118× · DOI ↗ · more like this ·

There has been a great deal of recent research into word sense disambiguation, particularly since the inception of the Senseval evaluation exercises. Because a word often has more than one meaning, resolving word sense ambiguity could benefit applications that need some level of semantic interpretation of language input. A major problem is that the accuracy of word sense disambiguation systems is strongly dependent on the quantity of manually sense-tagged data available, and even the best systems, when tagging every word token in a document, perform little better than a simple heuristic that guesses the first, or predominant, sense of a word in all contexts. The success of this heuristic is due to the skewed nature of word sense distributions. Data for the heuristic can come from either dictionaries or a sample of sense-tagged data. However, there is a limited supply of the latter, and…

Robust Grammatical Analysis for Spoken Dialogue Systems

Gertjan van Noord, Gosse Bouma, Rob Koeling +1 more

cs.CL · 25 Jun 1999 · cited 113× · arXiv ↗ · more like this ·

We argue that grammatical analysis is a viable alternative to concept spotting for processing spoken input in a practical spoken dialogue system. We discuss the structure of the grammar, and a model for robust parsing which combines linguistic sources of information and statistical sources of information. We discuss test results suggesting that grammatical processing allows fast and accurate processing of spoken input.

Chunking with maximum entropy models

Rob Koeling

Artificial Intelligence · 2000 · cited 84× · DOI ↗ · more like this ·

In this paper I discuss a first attempt to create a text chunker using a Maximum Entropy model. The first experiments, implementing classifiers that tag every word in a sentence with a phrase-tag using very local lexical information, part-of-speech tags and phrase tags of surrounding words, give encouraging results.

Domain-specific sense distributions and predominant sense acquisition

Rob Koeling, Diana McCarthy, John Carroll

Artificial Intelligence · Figshare · 2005 · cited 78× · DOI ↗ · more like this ·

Distributions of the senses of words are often highly skewed. This fact is exploited by word sense disambiguation (WSD) systems which back off to the predominant sense of a word when contextual clues are not strong enough. The domain of a document has a strong influence on the sense distribution of words, but it is not feasible to produce large manually annotated corpora for every domain of interest. In this paper we describe the construction of three sense annotated corpora in different domains for a sample of English words. We apply an existing method for acquiring predominant sense information automatically from raw text, and for our sample demonstrate that (1) acquiring such information automatically from a mixed-domain corpus is more accurate than deriving it from SemCor, and (2) acquiring it automatically from text in the same domain as the target domain performs best by a large…

Comparing grammar-based and robust approaches to speech understanding: a case study

Sylvia Knight, Genevieve Gorrell, Manny Rayner +3 more

Artificial Intelligence · 3 Sep 2001 · cited 60× · DOI ↗ · more like this ·

Previous work has demonstrated the success of statistical language models when enough training data is available [1], but despite that, grammar-based systems are proving the preferred choice in successful commercial systems such as HeyAnita [2], BeVocal [3] and Tellme [4], largely due to the difficulty involved in obtaining a corpus of training data. Here we trained an SLM on data obtained using a grammar-based system and compared the performance of the two systems with regards to recognition. We also parsed the output of the SLM using a robust parser and compared the accuracy of the semantic output of the systems. The SLM/robust parser showed considerable improvement on unconstrained input, and similar precision/recall (per slot value) on utterances provided by trained users. 1.

What does validation of cases in electronic record databases mean? The potential contribution of free text

Amanda Nicholson, A. Rosemary Tate, Rob Koeling +1 more

Artificial Intelligence · Pharmacoepidemiology and Drug Safety · 18 Jan 2011 · cited 48× · DOI ↗ · more like this ·

Electronic health records are increasingly used for research. The definition of cases or endpoints often relies on the use of coded diagnostic data, using a pre-selected group of codes. Validation of these cases, as 'true' cases of the disease, is crucial. There are, however, ambiguities in what is meant by validation in the context of electronic records. Validation usually implies comparison of a definition against a gold standard of diagnosis and the ability to identify false negatives ('true' cases which were not detected) as well as false positives (detected cases which did not have the condition). We argue that two separate concepts of validation are often conflated in existing studies. Firstly, whether the GP thought the patient was suffering from a particular condition (which we term confirmation or internal validation) and secondly, whether the patient really had the condition…

Applying System Combination to Base Noun Phrase Identification

Erik F. Tjong Kim Sang, Walter Daelemans, Herve Dejean +4 more

cs.CL · 17 Aug 2000 · cited 48× · arXiv ↗ · more like this ·

We use seven machine learning algorithms for one task: identifying base noun phrases. The results have been processed by different system combination methods and all of these outperformed the best individual result. We have applied the seven learners with the best combinator, a majority vote of the top five systems, to a standard data set and managed to improve the best published result for this data set.

Using automatically acquired predominant senses for word sense disambiguation

Diana McCarthy, Rob Koeling, Julie Weeds +1 more

Artificial Intelligence · Figshare · 2004 · cited 32× · OpenAlex ↗ · more like this ·

In word sense disambiguation (WSD), the heuristic of choosing the most common sense is extremely powerful because the distribution of the senses of a word is often skewed. The first (or predominant) sense heuristic assumes the availability of handtagged data. Whilst there are hand-tagged corpora available for some languages, these are relatively small in size and many word forms either do not occur, or occur infrequently. In this paper we investigate the performance of an unsupervised first sense heuristic where predominant senses are acquired automatically from raw text. We evaluate on both the SENSEVAL-2 and SENSEVAL-3 English allwords data. For accurate WSD the first sense heuristic should be used only as a back-off, where the evidence from the context is not strong enough. In this paper however, we examine the performance of the automatically acquired first sense in isolation since…

Sequence-Based Extractive Summarisation for Scientific Articles

Daniel Kershaw, Rob Koeling

cs.CL · 7 Apr 2022 · cited 17× · arXiv ↗ · more like this ·

This paper presents the results of research on supervised extractive text summarisation for scientific articles. We show that a simple sequential tagging model based only on the text within a document achieves high results against a simple classification model. Improvements can be achieved through additional sentence-level features, though these were minimal. Through further analysis, we show the potential of the sequential model relying on the structure of the document depending on the academic discipline which the document is from.

Sussx

Rob Koeling, Diana McCarthy

Artificial Intelligence · 2007 · cited 16× · DOI ↗ · more like this ·

We introduced a method for discovering the predominant sense of words automatically using raw (unlabelled) text in (McCarthy et al., 2004) and participated with this system in Senseval3. Since then, we worked on further developing ideas to improve upon the base method. In the current paper we target two areas where we believe there is potential for improvement. In the first one we address the finegrained structure of WordNet's (wn) sense inventory (i.e. the topic of the task in this particular track). The second issue we address here, deals with topic domain specilisation of the base method.

Grammatical analysis in the OVIS spoken-dialogue system

Mark-Jan Nederhof, Gosse Bouma, Rob Koeling +1 more

cmp-lg · May 1997 · cited 16× · arXiv ↗ · more like this ·

We argue that grammatical processing is a viable alternative to concept spotting for processing spoken input in a practical dialogue system. We discuss the structure of the grammar, the properties of the parser, and a method for achieving robustness. We discuss test results suggesting that grammatical processing allows fast and accurate processing of spoken input.

Estimating and exploiting the entropy of sense distributions

Peng Jin, Diana McCarthy, Rob Koeling +1 more

Artificial Intelligence · Figshare · 2009 · cited 10× · DOI ↗ · more like this ·

Word sense distributions are usually skewed. Predicting the extent of the skew can help a word sense disambiguation (WSD) system determine whether to consider evidence from the local context or apply the simple yet effective heuristic of using the first (most frequent) sense. In this paper, we propose a method to estimate the entropy of a sense distribution to boost the precision of a first sense heuristic by restricting its application to words with lower entropy. We show on two standard datasets that automatic prediction of entropy can increase the performance of an automatic first sense heuristic.

Gloss-Based Semantic Similarity Metrics for Predominant Sense Acquisition

Ryu Iida, Diana McCarthy, Rob Koeling

Artificial Intelligence · International Joint Conference on Natural Language Processing · 2008 · cited 8× · OpenAlex ↗ · more like this ·

In recent years there have been various approaches aimed at automatic acquisition of predominant senses of words. This information can be exploited as a powerful backoff strategy for word sense disambiguation given the zipfian distribution of word senses. Approaches which do not require manually sense-tagged data have been proposed for English exploiting lexical resources available, notably WordNet. In these approaches distributional similarity is coupled with a semantic similarity measure which ties the distributionally related words to the sense inventory. The semantic similarity measures that have been used have all taken advantage of the hierarchical information in WordNet. We investigate the applicability to Japanese and demonstrate the feasibility of a measure which uses only information in the dictionary definitions, in contrast with previous work on English which uses…

Learning Computational Grammars

John Nerbonne, Anja Belz, Nicola Cancedda +7 more

cs.CL · 15 Jul 2001 · cited 8× · arXiv ↗ · more like this ·

This paper reports on the "Learning Computational Grammars" (LCG) project, a postdoc network devoted to studying the application of machine learning techniques to grammars suitable for computational use. We were interested in a more systematic survey to understand the relevance of many factors to the success of learning, esp. the availability of annotated data, the kind of dependencies in the data, and the availability of knowledge bases (grammars). We focused on syntax, esp. noun phrase (NP) syntax.

Corpus Annotation as a Scientific Task

Donia Scott, Rossano Barone, Rob Koeling

Artificial Intelligence · Figshare · May 2012 · cited 7× · DOI ↗ · more like this ·

Annotation studies in CL are generally unscientific: they are mostly not reproducible, make use of too few (and often non-independent) annotators and use guidelines that are often something of a moving target. Additionally, the notion of ‘expert annotators’ invariably means only that the annotators have linguistic training. While this can be acceptable in some special contexts, it is often far from ideal. This is particularly the case when subtle judgements are required or when, as increasingly, one is making use of corpora originating from technical texts that have been produced by, and intended to be consumed by, an audience of technical experts in the field. We outline a more rigorous approach to collecting human annotations, using as our example a study designed to capture judgements on the meaning of hedge words in medical records.

Automatic identification of infrequent word senses

Diana McCarthy, Rob Koeling, Julie Weeds +1 more

Artificial Intelligence · Figshare · 2004 · cited 6× · DOI ↗ · more like this ·

In this paper we show that an unsupervised method for ranking word senses automatically can be used to identify infrequently occurring senses. We demonstrate this using a ranking of noun senses derived from the BNC and evaluating on the sense-tagged text available in both SemCor and the SENSEVAL-2 English all-words task. We show that the method does well at identifying senses that do not occur in a corpus, and that those that are erroneously filtered but do occur typically have a lower frequency than the other senses. This method should be useful for word sense disambiguation systems, allowing effort to be concentrated on more frequent senses; it may also be useful for other tasks such as lexical acquisition. Whilst the results on balanced corpora are promising, our chief motivation for the method is for application to domain specific text. For text within a particular domain many…

WASPBENCH

Adam Kilgarriff, Roger Evans, Rob Koeling +2 more

Artificial Intelligence · 2003 · cited 6× · DOI ↗ · more like this ·

Human Language Technologies (HLT) need dictionaries, to tell them what words mean and how they behave. People making dictionaries (lexicographers) need HLT, to help them identify how words behave so they can make better dictionaries. Thus a potential for synergy exists across the range of lexical data - in the construction of headword lists, for spelling correction, phonetics, morphology and syntax, but nowhere more than for semantics, and in particular the vexed question of how a word's meaning should be analysed into distinct senses. HLT needs all the help it can get from dictionaries, because it is a very hard problem to identify which meaning of a word applies. Lexicographers need all the help they can get because the analysis of meaning is the second hardest part of their job (Kilgarriff, 1998), it occupies a large share of their working hours, and it is one where, currently, they…

An evaluation of a lexicographer's workbench

Rob Koeling, Adam Kilgarriff, David Tugwell +1 more

Artificial Intelligence · 2003 · cited 4× · DOI ↗ · more like this ·

NLP system developers and corpus lexicographers would both benefit from a tool for finding and organizing the distinctive patterns of use of words in texts. Such a tool would be an asset for both language research and lexicon development, particularly for lexicons for Machine Translation (MT). We have developed the WASPBENCH, a tool that (1) presents a "word sketch", a summary of the corpus evidence for a word, to the lexicographer; (2) supports the lexicographer in analysing the word into its distinct meanings and (3) uses the lexicographer's analysis as the input to a state-of-the-art word sense disambiguation algorithm, the output of which is a "word expert" for the word which can then disambiguate new instances of the word. In this paper we describe a set of evaluation experiments, designed to establish whether waspbench can be used to save time and improve performance in the…

Cross-Language Acquisition of Semantic Models for Verbal Predicates

Jordi Atserias, Bernardo Magnini, Octavian Popescu +5 more

Artificial Intelligence · Figshare · May 2004 · cited 2× · DOI ↗ · more like this ·

This paper presents a semantic-driven methodology for the automatic acquisition of verbal models. Our approach relies strongly on the semantic generalizations allowed by already existing resources (e.g. Domain labels, Named Entity categories, concepts in the SUMO ontology, etc). Several experiments have been carried out using comparable corpora in four languages (Italian, Spanish, Basque and English) and two domains (FINANCE and SPORT) showing that the semantic patterns acquired can be general enough to be ported from one language to the other language. 1.

Elsevier OA CC-By Corpus

Daniel Kershaw, Rob Koeling

cs.CL · August 2020 · cited 1× · arXiv ↗ · more like this ·

We introduce the Elsevier OA CC-BY corpus. This is the first open corpus of Scientific Research papers which has a representative sample from across scientific disciplines. This corpus not only includes the full text of the article, but also the metadata of the documents, along with the bibliographic information for each reference.

Search in Elsevier Research Products: Current State and Lessons to be Learned

Rob Koeling

Artificial Intelligence · SSRN Electronic Journal · 25 Nov 2019 · cited 0× · OpenAlex ↗ · more like this ·

In this work we report on an analysis of the current user experience for search in a number of Elsevier products from a Natural Language Processing (NLP) point of view. This is an exploration of search behaviour and identifying where established NLP insights could alleviate the most pressing issues. In our analysis we identify a number of clusters of search patterns and explore the possibilities for improving the response given the user's search intent. We are also interested in consistency in the search experience across Elsevier platforms and relate to the higher expectations that have been set in the outside world in recent years. Finally, we hypothesise how a more sophisticated way of ingesting our content can help to look beyond indexing on the level of isolated tokens, in order to have more context available to link a search query to the most suitable content.

From predicting predominant senses to local context for word sense disambiguation

Rob Koeling, Diana McCarthy

Artificial Intelligence · 2008 · cited 0× · DOI ↗ · more like this ·

Recent work on automatically predicting the predominant sense of a word has proven to be promising (McCarthy et al., 2004). It can be applied (as a first sense heuristic) to Word Sense Disambiguation (WSD) tasks, without needing expensive hand-annotated data sets. Due to the big skew in the sense distribution of many words (Yarowsky and Florian, 2002), the First Sense heuristic for WSD is often hard to beat. However, the local context of an ambiguous word can give important clues to which of its senses was intended. The sense ranking method proposed by McCarthy et al. (2004) uses a distributional similarity thesaurus. The k nearest neighbours in the thesaurus are used to establish the predominant sense of a word. In this paper we report on a first investigation on how to use the grammatical relations the target word is involved with, in order to select a subset of the neighbours from…

Julie Weeds

Diana Mccarthy, Rob Koeling, John Carroll

Artificial Intelligence · 2007 · cited 0× · OpenAlex ↗ · more like this ·

There has been a great deal of recent research into word sense disambiguation, particularly since the inception of the Senseval evaluation exercises. Because a word often has more than one meaning, resolving word sense ambiguity could benefit applications that need some level of semantic interpretation of language input. A major problem is that the accuracy of word sense disambiguation systems is strongly dependent on the quantity of manually sense-tagged data available, and even the best systems, when tagging every word token in a document, perform little better than a simple heuristic that guesses the first, or predominant, sense of a word in all contexts. The success of this heuristic is due to the skewed nature of word sense distributions. Data for the heuristic can come from either dictionaries or a sample of sensetagged data. However, there is a limited supply of the latter, and…

Ranking WordNet Senses Automatically

Diana McCarthy, Rob Koeling, Julie Weeds

Artificial Intelligence · 2004 · cited 0× · OpenAlex ↗ · more like this ·

In word sense disambiguation (WSD), the heuristic of choosing the first listed sense in a dictionary is often hard to beat, especially by systems that do not exploit handtagged training data. The problem with using the first sense heuristic, aside from the fact that it does not take surrounding context into account, is that it assumes some quantity of hand-tagged data. Whilst there are some hand-tagged corpora available for some languages, one would expect the frequency distribution of the senses of words to depend on the genre and domain of the text under consideration. For example, one would expect a different predominant sense for star if one were looking at scientific astronomy reports compared with popular news. We present work on the use of the WordNet similarity package and a thesaurus automatically acquired from raw textual corpora to rank WordNet noun senses automatically. The…