arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Adam Janin

42 papers · 1995–2022

1995: 11996: 01997: 01998: 01999: 42000: 02001: 22002: 22003: 12004: 22005: 02006: 02007: 22008: 22009: 22010: 62011: 52012: 02013: 22014: 12015: 12016: 62017: 12018: 12019: 02020: 02021: 02022: 1
1995peak 6 in 20102022

Frequent co-authors

Papers

showing 1–25 of 42 RSS

The ICSI Meeting Corpus

Adam Janin, Don Baron, Jane A. Edwards +8 more

Artificial Intelligence · 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03). · · cited 672× · DOI ↗ · more like this ·

We have collected a corpus of data from natural meetings that occurred at the International Computer Science Institute (ICSI) in Berkeley, California over the last three years. The corpus contains audio recorded simultaneously from head-worn and table-top microphones, word-level transcripts of meetings, and various metadata on participants, meetings, and hardware. Such a corpus supports work in automatic speech recognition, noise robustness, dialog modeling, prosody, rich transcription, information retrieval, and more. We present details on the contents of the corpus, as well as rationales for the decisions that led to its configuration. The corpus were delivered to the Linguistic Data Consortium (LDC).

Calibration of head-mounted displays for augmented reality applications

Adam Janin, David Mizell, Thomas P. Caudell

Computer Vision and Pattern Recognition · · cited 162× · DOI ↗ · more like this ·

The authors have developed "augmented reality" technology, consisting of a see-through head-mounted display, a robust, accurate position/orientation sensor, and their supporting electronics and software. Their primary goal is to apply this technology to touch labor manufacturing processes, enabling a factory worker to view index markings or instructions as if they were painted on the surface of a workpiece. In order to accurately project graphics onto specific points of a workpiece, it is necessary to have the coordinates of the workpiece, the display's virtual screen, the position sensor, and the user's eyes in the same coordinate system. The linear transformation and projection of each point to be displayed from world coordinates to virtual screen coordinates are described, and the experimental procedures for determining the correct values of the calibration parameters are…

The meeting project at ICSI

Nelson Morgan, Don Baron, Jane A. Edwards +6 more

Artificial Intelligence · Defense Technical Information Center (DTIC) · · cited 129× · DOI ↗ · more like this ·

In collaboration with colleagues at UW, OGI, IBM, and SRI, we are developing technology to process spoken language from informal meetings. The work includes a substantial data collection and transcription effort, and has required a nontrivial degree of infrastructure development. We are undertaking this because the new task area provides a significant challenge to current HLT capabilities, while offering the promise of a wide range of potential applications. In this paper, we give our vision of the task, the challenges it represents, and the current state of our development, with particular attention to automatic transcription.

The ICSI Meeting Project: Resources and Research

Adam Janin, J. Ang, Sonali Bhagat +9 more

Artificial Intelligence · PUB – Publications at Bielefeld University (Bielefeld University) · · cited 96× · OpenAlex ↗ · more like this ·

This paper provides a progress report on ICSI’s Meeting Project, including both the data collected and annotated as part of the project, as well as the research lines such materials support. We include a general description of the official “ICSI Meeting Corpus”, as currently available through the Linguistic Data Consortium, discuss some of the existing and planned annotations which augment the basic transcripts provided there, and describe several research efforts that make use of these materials. The corpus supports wideranging efforts, from low-level processing of the audio signal (including automatic speech transcription, speaker tracking, and work on far-field acoustics) to higher-level analyses of meeting structure, content, and interactions (such as topic and sentence segmentation, and automatic detection of dialogue acts and meeting “hot spots”). 1.

Meetings about meetings: research at ICSI on speech in multiparty conversations

N. Morgan, Don Baron, Sonali Bhagat +9 more

Artificial Intelligence · 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03). · · cited 67× · DOI ↗ · more like this ·

In early 2001, we reported (at the Human Language Technology meeting) the early stages of an ICSI (International Computer Science Institute) project on processing speech from meetings (in collaboration with other sites, principally SRI, Columbia, and UW). We report our progress from the first few years of this effort, including: the collection and subsequent release of a 75-meeting corpus (over 70 meeting-hours and up to 16 channels for each meeting); the development of a prosodic database for a large subset of these meetings, and its subsequent use for punctuation and disfluency detection; the development of a dialog annotation scheme and its implementation for a large subset of the meetings; and the improvement of both near-mic and far-mic speech recognition results for meeting speech test sets.

The TAO of ATWV: Probing the mysteries of keyword search performance

Steven Wegmann, Arlo Faria, Adam Janin +2 more

Artificial Intelligence · · cited 48× · DOI ↗ · more like this ·

In this paper we apply diagnostic analysis to gain a deeper understanding of the performance of the the keyword search system that we have developed for conversational telephone speech in the IARPA Babel program. We summarize the Babel task, its primary performance metric, “actual term weighted value” (ATWV), and our recognition and keyword search systems. Our analysis uses two new oracle ATWV measures, a bootstrap-based ATWV confidence interval, and includes a study of the underpinnings of the large ATWV gains due to system combination. This analysis quantifies the potential ATWV gains from improving the number of true hits and the overall quality of the detection scores in our system's posting lists. It also shows that system combination improves our systems' ATWV via a small increase in the number of true hits in the posting lists.

The ICSI RT-09 Speaker Diarization System

Gerald Friedland, Adam Janin, David Imseng +5 more

Artificial Intelligence · IEEE Transactions on Audio Speech and Language Processing · · cited 47× · DOI ↗ · more like this ·

The speaker diarization system developed at the International Computer Science Institute (ICSI) has played a prominent role in the speaker diarization community, and many researchers in the rich transcription community have adopted methods and techniques developed for the ICSI speaker diarization engine. Although there have been many related publications over the years, previous articles only presented changes and improvements rather than a description of the full system. Attempting to replicate the ICSI speaker diarization system as a complete entity would require an extensive literature review, and might ultimately fail due to component description version mismatches. This paper therefore presents the first full conceptual description of the ICSI speaker diarization system as presented to the National Institute of Standards Technology Rich Transcription 2009 (NIST RT-09) evaluation…

Multi-stream speech recognition: Ready for prime time?

Adam Janin, Daniel P. W. Ellis, Nelson Morgan

Artificial Intelligence · Columbia Academic Commons (Columbia University) · · cited 45× · DOI ↗ · more like this ·

Multi-stream and multi-band methods can improve the accuracy of speech recognition systems without overly increasing the complexity. However, they cannot be applied blindly. In this paper, we review our experience applying multi-stream and multiband methods to the Broadcast News corpus. We found that multi-stream systems using different acoustic front-ends provide a significant improvement over single stream systems. However, despite the fact that they have been successful on smaller tasks, we have not yet been able to show any gain using multi-band methods. We report various insights gained from the experience in applying these methods in a large-vocabulary task.

All for one: feature combination for highly channel-degraded speech activity detection

Martin Graciarena, Abeer Alwan, Dan Ellis +9 more

Signal Processing · · cited 42× · DOI ↗ · more like this ·

Speech activity detection (SAD) on channel transmissions is a critical preprocessing task for speech, speaker and language recognition or for further human analysis. This paper presents a feature combination approach to improve SAD on highly channel degraded speech as part of the Defense Advanced

The YLI-MED Corpus: Characteristics, Procedures, and Plans

Julia Bernd, Damian Borth, Benjamin Elizalde +7 more

cs.MM · · cited 25× · arXiv ↗ · more like this ·

The YLI Multimedia Event Detection corpus is a public-domain index of videos with annotations and computed features, specialized for research in multimedia event detection (MED), i.e., automatically identifying what's happening in a video by analyzing the audio and visual content. The videos indexed in the YLI-MED corpus are a subset of the larger YLI feature corpus, which is being developed by the International Computer Science Institute and Lawrence Livermore National Laboratory based on the Yahoo Flickr Creative Commons 100 Million (YFCC100M) dataset. The videos in YLI-MED are categorized as depicting one of ten target events, or no target event, and are annotated for additional attributes like language spoken and whether the video has a musical score. The annotations also include degree of annotator agreement and average annotator confidence scores for the event categorization of…

Multimodal location estimation on Flickr videos

Gerald Friedland, Jaeyoung Choi, Howard Lei +1 more

Computer Vision and Pattern Recognition · · cited 25× · DOI ↗ · more like this ·

The following article describes an approach to determine the geo-coordinates of the recording place of Flickr videos based on both textual metadata and visual cues. The system is tested on the MediaEval 2010 Placing Task evaluation data, which consists of 5091 unfiltered test videos. The system presented in this article is less complex, uses less training data, and is at the same time more accurate than the best system presented in the evaluation in August 2010. The performance peaks at being able to classify 14% of the videos with less than 10m accuracy. The article describes the realization of the system, analyses of the different uses of multimodal cues and gazetteer information.

Mutaphrase

Michael Ellsworth, Adam Janin

Artificial Intelligence · · cited 23× · DOI ↗ · more like this ·

We describe a preliminary version of Mutaphrase, a system that generates paraphrases of semantically labeled input sentences using the semantics and syntax encoded in FrameNet, a freely available lexicosemantic database. The algorithm generates a large number of paraphrases with a wide range of syntactic and semantic distances from the input. For example, given the input "I like eating cheese", the system outputs the syntactically distant "Eating cheese is liked by me", the semantically distant "I fear sipping juice", and thousands of other sentences. The wide range of generated paraphrases makes the algorithm ideal for a range of statistical machine learning problems such as machine translation and language modeling as well as other semanticsdependent tasks such as query and language generation.

The IBM 2009 GALE Arabic speech transcription system

Brian Kingsbury, Hagen Soltau, George Saon +6 more

Artificial Intelligence · · cited 22× · DOI ↗ · more like this ·

We describe the Arabic broadcast transcription system fielded by IBM in the GALE Phase 4 machine translation evaluation. Key advances over our Phase 3.5 system include improvements to context-dependent modeling in vowelized Arabic acoustic models; the use of neural-network features provided by the International Computer Science Institute; Model M language models; a neural network language model that uses syntactic and morphological features; and improvements to our system combination strategy. These advances were instrumental in achieving a word error rate of 8.9% on the Phase 4 evaluation set, and an absolute improvement of 1.6% word error rate over our 2008 system on the unsequestered Phase 3.5 evaluation data.

Opportunities and challenges of parallelizing speech recognition

Jike Chong, Gerald Friedland, Adam Janin +2 more

Artificial Intelligence · USENIX conference on Hot topics in parallelism · · cited 18× · OpenAlex ↗ · more like this ·

Automatic speech recognition enables a wide range of current and emerging applications such as automatic transcription, multimedia content analysis, and natural human-computer interfaces. This article provides a glimpse of the opportunities and challenges that parallelism provides for automatic speech recognition and related application research from the point of view of speech researchers. The increasing parallelism in computing platforms opens three major possibilities for speech recognition systems: improving recognition accuracy in non-ideal, everyday noisy environments; increasing recognition throughput in batch processing of speech data; and reducing recognition latency in realtime usage scenarios. We describe technical challenges, approaches we’ve taken, and possible directions for future research to guide the design of efficient parallel software and hardware infrastructures. 1

Joke-o-mat

Gerald Friedland, Luke Gottlieb, Adam Janin

Computer Vision and Pattern Recognition · · cited 17× · DOI ↗ · more like this ·

This paper summarizes our contribution to the Yahoo! task of the ACM Multimedia Grand Challenge. This challenge asks for the robust automatic segmentation of videos according to "narrative themes". Based on the automatic segmentation methods presented in [1] and partly [2], we describe a system to navigate Seinfeld episodes based on automatic segmentation of the audio track only. The system distinguishes laughter, music, and other noise as well as speech segments. Speech segments are identified against pre-trained speaker models of the actors. Given this segmentation and the artistic production rules that underlie the genre situation comedy and Seinfeld in particular, the system enables a user to browse an episode by scene, by punchline, and by dialog segments. The themes can be filtered by the main actors, e.g. the user can select to see only punchlines by Jerry and Kramer. Based on…

Video2GPS

Gerald Friedland, Jaeyoung Choi, Adam Janin

Computer Vision and Pattern Recognition · · cited 15× · DOI ↗ · more like this ·

The following article describes our demo of an approach to determine the geo-coordinates of the recording place of Flickr videos based on both textual metadata and visual cues. The underlying system has been tested on the MediaEval 2010 Placing Task evaluation data, which consists of 5091 unfiltered test videos is able to classify 14% of the videos to within an accuracy of 10m.

Parallelizing Speaker-Attributed Speech Recognition for Meeting Browsing

Gerald Friedland, Jike Chong, Adam Janin

Artificial Intelligence · · cited 13× · DOI ↗ · more like this ·

The following article presents an application for browsing meeting recordings by speaker and keyword which we call the Meeting Diarist. The goal of the system is to enable browsing of the content with rich meta-data in a graphical user interface shortly after the end of meeting, even when the application runs on a contemporary laptop. We there-fore developed novel parallel methods for speaker diarization and multi-hypothesis speech recognition that are optimized to run on multicore and many core architectures. This paper presents the underlying parallel speaker diarization and speech recognition realizations, a comparison of results based on NIST RT07 evaluation data, and a description of the final application.

Application-Independent and Integration-Friendly Natural Language Understanding

Manfred Eppe, Sean Trott, Vivek Raghuram +2 more

Artificial Intelligence · EPiC series in computing · · cited 12× · DOI ↗ · more like this ·

Natural Language Understanding (NLU) has been a long-standing goal of AI and many related fields, but it is often dismissed as very hard to solve. NLU is required complex flexible systems that take action without further human intervention. This inherently involves strong semantic (meaning) capabilities to parse queries and commands correctly and with high confidence, because an error by a robot or automated vehicle could be disastrous. We describe an implemented general framework, the ECG2 system, that supports the deployment of NLU systems over a wide range of application domains. The framework is based on decades of research on embodied action-oriented semantics and efficient computational realization of a deep semantic analyzer (parser). This makes it linguistically much more flexible, general and reliable than existing shallow approaches that process language without considering…

DCAR: A Discriminative and Compact Audio Representation for Audio Processing

Liping Jing, Bo Liu, Jaeyoung Choi +4 more

Signal Processing · IEEE Transactions on Multimedia · · cited 12× · DOI ↗ · more like this ·

This paper presents a novel two-phase method for audio representation, discriminative and compact audio representation (DCAR), and evaluates its performance at detecting events and scenes in consumer-produced videos. In the first phase of DCAR, each audio track is modeled using a Gaussian mixture model (GMM) that includes several components to capture the variability within that track. The second phase takes into account both global structure and local structure. In this phase, the components are rendered more discriminative and compact by formulating an optimization problem on a Grassmannian manifold. The learned components can effectively represent the structure of audio. Our experiments used the YLI-MED and DCASE Acoustic Scenes datasets. The results show that variants on the proposed DCAR representation consistently outperform four popular audio representations (mv-vector, i-vector…

Joke-o-Mat HD

Adam Janin, Luke Gottlieb, Gerald Friedland

Signal Processing · · cited 9× · DOI ↗ · more like this ·

Joke-o-mat HD is a system that allows a user to navigate sitcoms (such as Seinfeld) by "narrative themes", including scenes, punchlines, and dialog segments. The themes can be filtered by the main actors and by keyword. For example, the user can select to see only punchlines by Kramer that contain the word "armoire". The system infers the narrative themes using segmentation of the audio track into laughter, actors, words, and music. The segmentation can be generated either by an expert annotator, via automatic methods, or by exploiting human derived (HD) "found" data such as fan-generated scripts and closed captions. We demonstrate browsing one episode of Seinfeld using all three methods of generating segmentations.

The SRI-ICSI Spring 2007 meeting and lecture recognition system

Andreas Stolcke, Xavier Anguera, Kofi Boakye +4 more

Signal Processing · · cited 9× · OpenAlex ↗ · more like this ·

Abstract. We describe the latest version of the SRI-ICSI meeting and lecture recognition system, as was used in the NIST RT-07 evaluations, highlighting improvements made over the last year. Changes in the acoustic preprocessing include updated beamforming software for processing of multiple distant mi-crophones, and various adjustments to the speech segmenter for close-talking microphones. Acoustic models were improved by the combined use of neural-net-estimated phone posterior features, discriminative feature transforms trained with fMPE-MAP, and discriminative Gaussian estimation using MPE-MAP, as well as model adaptation specifically to nonnative and non-American speakers. The net effect of these enhancements was a 14-16 % relative error reduction on distant microphones, and a 16-17 % error reduction on close-talking microphones. Also, for the first time, we report results on a new…

<title>Videometric head tracker for augmented reality applications</title>

Adam Janin, Karel Zikan, David Mizell +2 more

Computer Vision and Pattern Recognition · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · · cited 9× · DOI ↗ · more like this ·

For the past three years, we have been developing augmented reality technology for application to a variety of touch labor tasks in aircraft manufacturing and assembly. The system would be worn by factory workers to provide them with better-quality information for performing their tasks than was previously available. Using a see-through head-mounted display (HMD) whose optics are set at a focal length of about 18 in., the display and its associated head tracking system can be used to superimpose and stabilize graphics on the surface of a work piece. This technology would obviate many expensive marking systems now used in aerospace manufacturing. The most challenging technical issue with respect to factory applications of AR is head position and orientation tracking. It requires high accuracy, long- range tracking in a high-noise environment. The approach we have chosen uses a head-…

A Discriminative and Compact Audio Representation for Event Detection

Liping Jing, Bo Liu, Jaeyoung Choi +4 more

Signal Processing · · cited 8× · DOI ↗ · more like this ·

This paper presents a novel two-phase method for audio representation: Discriminative and Compact Audio Representation (DCAR). In the first phase, each audio track is modeled using a Gaussian mixture model (GMM) that includes several components to capture the variability within that track. The second phase takes into account both global structure and local structure. In this phase, the components are rendered more discriminative and compact by formulating an optimization problem on Grassmannian manifolds, which we found represents the structure of audio effectively. Experimental results on the YLI-MED dataset show that the proposed DCAR representation consistently outperforms state-of-the-art audio representations: i-vector, mv-vector, and GMM.

User verification: Matching the uploaders of videos across accounts

Howard Lei, Jaeyoung Choi, Adam Janin +1 more

Information Systems · · cited 8× · DOI ↗ · more like this ·

This article presents an attempt to link the uploaders of videos based on the audio track of the videos. Using a subset of the MediaEval Placing Task's Flickr video set, which is labeled with the uploader's name, we conducted an experiment with a similar setup as a typical NIST speaker recognition evaluation run. Based on the assumption that the audio might be matched in various ways (speaker, channel, environmental noise, etc.), we trained one of ICSI's simplified speaker recognition systems on the audio tracks of the Flickr videos. Note that since the selection of videos is essentially random, the audio track can contain any sounds. We obtain an equal error rate of 36.7% on 312 videos with 11,550 trials. The result has implications for audio research, security applications, and raises privacy concerns.

Reducing errors by increasing the error rate: MLP Acoustic Modeling for Broadcast News Transcription

Nelson Morgan, Daniel P. W. Ellis, Eric Fosler‐Lussier +2 more

Artificial Intelligence · Columbia Academic Commons (Columbia University) · · cited 8× · DOI ↗ · more like this ·

We describe some aspects of a Broadcast News recognition system based on hybrid HMM/MLP acoustic modeling. These include the use of novel 'modulation spectrogram' features which are combined with conventional models at the posterior probability level, some experiments with nonlinear segment normalization, and an investigation of the interaction of model size and training set size for an multilayer perceptron (MLP) acoustic classifier. We also report preliminary results of incorporating gender-dependence into this system.