arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Mark Cartwright

66 papers · 1985–2026

1985: 11986: 01987: 01988: 01989: 01990: 01991: 01992: 01993: 61994: 01995: 01996: 11997: 01998: 01999: 02000: 02001: 12002: 12003: 02004: 12005: 02006: 02007: 32008: 02009: 02010: 12011: 22012: 12013: 02014: 42015: 32016: 72017: 22018: 52019: 82020: 22021: 42022: 32023: 32024: 22025: 12026: 4
1985peak 8 in 20192026

Frequent co-authors

Papers

showing 1–25 of 66 RSS

Scaper: A library for soundscape synthesis and augmentation

Justin Salamon, Duncan MacConnell, Mark Cartwright +2 more

Signal Processing · · cited 209× · DOI ↗ · more like this ·

Sound event detection (SED) in environmental recordings is a key topic of research in machine listening, with applications in noise monitoring for smart cities, self-driving cars, surveillance, bioa-coustic monitoring, and indexing of large multimedia collections. Developing new solutions for SED often relies on the availability of strongly labeled audio recordings, where the annotation includes the onset, offset and source of every event. Generating such precise annotations manually is very time consuming, and as a result existing datasets for SED with strong labels are scarce and limited in size. To address this issue, we present Scaper, an open-source library for soundscape synthesis and augmentation. Given a collection of iso-lated sound events, Scaper acts as a high-level sequencer that can generate multiple soundscapes from a single, probabilistically defined, “specification”. To…

Per-Channel Energy Normalization: Why and How

Vincent Lostanlen, Justin Salamon, Mark Cartwright +4 more

Signal Processing · IEEE Signal Processing Letters · · cited 101× · DOI ↗ · more like this ·

In the context of automatic speech recognition and acoustic event detection, an adaptive procedure named per-channel energy normalization (PCEN) has recently shown to outperform the pointwise logarithm of mel-frequency spectrogram (logmelspec) as an acoustic frontend. This letter investigates the adequacy of PCEN for spectrogram-based pattern recognition in far-field noisy recordings, both from theoretical and practical standpoints. First, we apply PCEN on various datasets of natural acoustic environments and find empirically that it Gaussianizes distributions of magnitudes while decorrelating frequency bands. Second, we describe the asymptotic regimes of each component in PCEN: temporal integration, gain control, and dynamic range compression. Third, we give practical advice for adapting PCEN parameters to the temporal properties of the noise to be mitigated, the signal to be enhanced…

Proof of Concept: A PhRMA Position Paper With Recommendations for Best Practice

Mark E. Cartwright, Steven E. Cohen, Joseph C. Fleishaker +4 more

Computational Theory and Mathematics · Clinical Pharmacology & Therapeutics · · cited 63× · DOI ↗ · more like this ·

Proof of concept (POC) may be defined as the earliest point in the drug development process at which the weight of evidence suggests that it is "reasonably likely" that the key attributes for success are present and the key causes of failure are absent. POC is multidimensional but is focused on attributes that, if not addressed, represent a threat to the success of the project in crucial areas such as safety, efficacy, pharmaceutics, and commercial and regulatory issues. The appropriate weight of evidence is assessed through the use of mathematical models and by evaluating the consequences of advancing a candidate drug that is not safe, effective, or commercially viable, vs. failing to advance a candidate that possesses these attributes. Tools for POC include biomarkers, targeted populations, pharmacokinetic (PK)/pharmacodynamic (PD) modeling, simulation, and adaptive study designs.…

Seeing Sound

Mark Cartwright, Ayanna Seals, Justin Salamon +6 more

Signal Processing · Proceedings of the ACM on Human-Computer Interaction · · cited 62× · DOI ↗ · more like this ·

Audio annotation is key to developing machine-listening systems; yet, effective ways to accurately and rapidly obtain crowdsourced audio annotations is understudied. In this work, we seek to quantify the reliability/redundancy trade-off in crowdsourced soundscape annotation, investigate how visualizations affect accuracy and efficiency, and characterize how performance varies as a function of audio characteristics. Using a controlled experiment, we varied sound visualizations and the complexity of soundscapes presented to human annotators. Results show that more complex audio scenes result in lower annotator agreement, and spectrogram visualizations are superior in producing higher quality annotations at lower cost of time and human labor. We also found recall is more affected than precision by soundscape complexity, and mistakes can be often attributed to certain sound event…

Fast and easy crowdsourced perceptual audio evaluation

Mark Cartwright, Bryan Pardo, Gautham J. Mysore +1 more

Signal Processing · · cited 60× · DOI ↗ · more like this ·

Automated objective methods of audio evaluation are fast, cheap, and require little effort by the investigator. However, objective evaluation methods do not exist for the output of all audio processing algorithms, often have output that correlates poorly with human quality assessments, and require ground truth data in their calculation. Subjective human ratings of audio quality are the gold standard for many tasks, but are expensive, slow, and require a great deal of effort to recruit subjects and run listening tests. Moving listening tests from the lab to the micro-task labor market of Amazon Mechanical Turk speeds data collection and reduces investigator effort. However, it also reduces the amount of control investigators have over the testing environment, adding new variability and potential biases to the data. In this work, we compare multiple stimulus listening tests performed in a…

SONYC Urban Sound Tagging (SONYC-UST): A Multilabel Dataset from an Urban Acoustic Sensor Network

Mark Cartwright, Ana Elisa Méndez Méndez, Jason Cramer +6 more

Signal Processing · Faculty Digital Archive (New York University Florence) · · cited 59× · DOI ↗ · more like this ·

SONYC Urban Sound Tagging (SONYC-UST) is a dataset for the development and evaluation of machine listening systems for realworld urban noise monitoring. It consists of 3068 audio recordings from the "Sounds of New York City" (SONYC) acoustic sensor network. Via the Zooniverse citizen science platform, volunteers tagged the presence of 23 fine-grained classes that were chosen in consultation with the New York City Department of Environmental Protection. These 23 fine-grained classes can be grouped into eight coarse-grained classes. In this work, we describe the collection of this dataset, metrics used to evaluate tagging systems, and the results of a simple baseline model.

SONYC-UST-V2: An Urban Sound Tagging Dataset with Spatiotemporal Context

Mark Cartwright, Jason Cramer, Ana Elisa Mendez Mendez +9 more

cs.SD · · cited 49× · arXiv ↗ · more like this ·

We present SONYC-UST-V2, a dataset for urban sound tagging with spatiotemporal information. This dataset is aimed for the development and evaluation of machine listening systems for real-world urban noise monitoring. While datasets of urban recordings are available, this dataset provides the opportunity to investigate how spatiotemporal metadata can aid in the prediction of urban sound tags. SONYC-UST-V2 consists of 18510 audio recordings from the "Sounds of New York City" (SONYC) acoustic sensor network, including the timestamp of audio acquisition and location of the sensor. The dataset contains annotations by volunteers from the Zooniverse citizen science platform, as well as a two-stage verification with our team. In this article, we describe our data collection procedure and propose evaluation metrics for multilabel classification of urban sound tags. We report the results of a…

Few-Shot Continual Learning for Audio Classification

Yu Wang, Nicholas J. Bryan, Mark Cartwright +2 more

Signal Processing · · cited 47× · DOI ↗ · more like this ·

Supervised learning for audio classification typically imposes a fixed class vocabulary, which can be limiting for real-world applications where the target class vocabulary is not known a priori or changes dynamically. In this work, we introduce a few-shot continual learning framework for audio classification, where we can continuously expand a trained base classifier to recognize novel classes based on only few labeled data at inference time. This enables fast and interactive model updates by end-users with minimal human effort. To do so, we leverage the dynamic few-shot learning technique and adapt it to a challenging multi-label audio classification scenario. We incorporate a recent state-of-the-art audio feature extraction model as a backbone and perform a comparative analysis of our approach on two popular audio datasets (ESC-50 and AudioSet). We conduct an in-depth evaluation to…

Crowdsourcing Multi-label Audio Annotation Tasks with Citizen Scientists

Mark Cartwright, Graham Dove, Ana Elisa Méndez Méndez +2 more

Signal Processing · · cited 46× · DOI ↗ · more like this ·

Annotating rich audio data is an essential aspect of training and evaluating machine listening systems. We approach this task in the context of temporally-complex urban soundscapes, which require multiple labels to identify overlapping sound sources. Typically this work is crowdsourced, and previous studies have shown that workers can quickly label audio with binary annotation for single classes. However, this approach can be difficult to scale when multiple passes with different focus classes are required to annotate data with multiple labels. In citizen science, where tasks are often image-based, annotation efforts typically label multiple classes simultaneously in a single pass. This paper describes our data collection on the Zooniverse citizen science platform, comparing the efficiencies of different audio annotation strategies. We compared multiple-pass binary annotation…

Open-Source Practices for Music Signal Processing Research: Recommendations for Transparent, Sustainable, and Reproducible Audio Research

Brian McFee, Jong Wook Kim, Mark Cartwright +3 more

Signal Processing · IEEE Signal Processing Magazine · · cited 44× · DOI ↗ · more like this ·

In the early years of music information retrieval (MIR), research problems were often centered around conceptually simple tasks, and methods were evaluated on small, idealized data sets. A canonical example of this is genre recognition-i.e., Which one of n genres describes this song?-which was often evaluated on the GTZAN data set (1,000 musical excerpts balanced across ten genres) [1]. As task definitions were simple, so too were signal analysis pipelines, which often derived from methods for speech processing and recognition and typically consisted of simple methods for feature extraction, statistical modeling, and evaluation. When describing a research system, the expected level of detail was superficial: it was sufficient to state, e.g., the number of mel-frequency cepstral coefficients used, the statistical model (e.g., a Gaussian mixture model), the choice of data set, and the…

VocalSketch

Mark Cartwright, Bryan Pardo

Signal Processing · · cited 39× · DOI ↗ · more like this ·

A natural way of communicating an audio concept is to imitate it with one's voice. This creates an approximation of the imagined sound (e.g. a particular owl's hoot), much like how a visual sketch approximates a visual concept (e.g a drawing of the owl). If a machine could understand vocal imitations, users could communicate with software in this natural way, enabling new interactions (e.g. programming a music synthesizer by imitating the desired sound with one's voice). In this work, we collect thousands of crowd-sourced vocal imitations of a large set of diverse sounds, along with data on the crowd's ability to correctly label these vocal imitations. The resulting data set will help the research community understand which audio concepts can be effectively communicated with this approach. We have released the data set so the community can study the related issues and build systems that…

Who calls the shots? Rethinking Few-Shot Learning for Audio

Yu Wang, Nicholas J. Bryan, Justin Salamon +2 more

cs.SD · · cited 30× · arXiv ↗ · more like this ·

Few-shot learning aims to train models that can recognize novel classes given just a handful of labeled examples, known as the support set. While the field has seen notable advances in recent years, they have often focused on multi-class image classification. Audio, in contrast, is often multi-label due to overlapping sounds, resulting in unique properties such as polyphony and signal-to-noise ratios (SNR). This leads to unanswered questions concerning the impact such audio properties may have on few-shot learning system design, performance, and human-computer interaction, as it is typically up to the user to collect and provide inference-time support set examples. We address these questions through a series of experiments designed to elucidate the answers to these questions. We introduce two novel datasets, FSD-MIX-CLIPS and FSD-MIX-SED, whose programmatic generation allows us to…

Active Learning for Efficient Audio Annotation and Classification with a Large Amount of Unlabeled Data

Yu Wang, Ana Elisa Méndez Méndez, Mark Cartwright +1 more

Signal Processing · · cited 26× · DOI ↗ · more like this ·

There are many sound classification problems that have target classes which are rare or unique to the context of the problem. For these problems, existing data sets are not sufficient and we must create new problem-specific datasets to train classification models. However, annotating a new dataset for every new problem is costly. Active learning could potentially reduce this annotation cost, but it has been understudied in the context of audio annotation. In this work, we investigate active learning to reduce the annotation cost of a sound classification dataset unique to a particular problem. We evaluate three certainty-based active learning query strategies and propose a new strategy: alternating confidence sampling. Using this strategy, we demonstrate reduced annotation costs when actively training models with both experts and non-experts, and we perform a qualitative analysis on 20k…

Voice Anonymization in Urban Sound Recordings

Alice Cohen-Hadria, Mark Cartwright, Brian McFee +1 more

Signal Processing · · cited 24× · DOI ↗ · more like this ·

Monitoring health and noise pollution in urban environments often entails deploying acoustic sensor networks to passively collect data in public spaces. Although spaces are technically public, people in the environment may not fully realize the degree to which they may be recorded by the sensor network, which may be perceived as a violation of expected privacy. Therefore, we propose a method to anonymize and blur the voices of people recorded in public spaces-a novel, yet increasingly important task as acoustic sensing becomes ubiquitous in sensor-equipped smart cities. This method is analogous to Google's face blurring in its Street View photographs, which arose from similar concerns in the visual domain. The proposed blurring method aims to anonymize voices by removing both the linguistic content and personal identity from voices, while preserving the rest of the acoustic scene.The…

EdgeL^3: Compressing L^3-Net for Mote Scale Urban Noise Monitoring

Sangeeta Kumari, Dhrubojyoti Roy, Mark Cartwright +2 more

Signal Processing · · cited 18× · DOI ↗ · more like this ·

Urban noise sensing in deeply embedded devices at the edge of the Internet of Things (IoT) is challenging not only because of the lack of sufficiently labeled training data but also because device resources are quite limited. Look, Listen, and Learn (L <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> ), a recently proposed state-of-the-art transfer learning technique, mitigates the first challenge by training self-supervised deep audio embeddings through binary Audio-Visual Correspondence (AVC), and the resulting embeddings can be used to train a variety of downstream audio classification tasks. However, with close to 4.7 million parameters, the multi-layer L <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> -Net CNN is still prohibitively expensive to be run on small edge devices…

MIXPLORATION

Mark Cartwright, Bryan Pardo, Josh Reiss

Signal Processing · · cited 17× · DOI ↗ · more like this ·

A typical audio mixer interface consists of faders and knobs that control the amplitude level as well as processing (e.g. equalization, compression and reverberation) parameters of individual tracks. This interface, while widely used and effective for optimizing a mix, may not be the best interface to facilitate exploration of different mixing options. In this work, we rethink the mixer interface, describing an alternative interface for exploring the space of possible mixes of four audio tracks. In a user study with 24 participants, we compared the effectiveness of this interface to the traditional paradigm for exploring alternative mixes. In the study, users responded that the proposed alternative interface facilitated exploration and that they considered the process of rating mixes to be beneficial.

Tricycle: Audio Representation Learning from Sensor Network Data Using Self-Supervision

Mark Cartwright, Aurora Cramer, Justin Salamon +1 more

Signal Processing · · cited 14× · DOI ↗ · more like this ·

Self-supervised representation learning with deep neural networks is a powerful tool for machine learning tasks with limited labeled data but extensive unlabeled data. To learn representations, self-supervised models are typically trained on a pretext task to predict structure in the data (e.g. audio-visual correspondence, short-term temporal sequence, word sequence) that is indicative of higher-level concepts relevant to a target, downstream task. Sensor networks are promising yet unexplored sources of data for self-supervised learning—they collect large amounts of unlabeled yet timestamped data over extended periods of time and typically exhibit long-term temporal structure (e.g., over hours, months, years) not observable at the short time scales previously explored in self-supervised learning (e.g., seconds). This structure can be present even in single-modal data and therefore could…

SynthAssist

Mark Cartwright, Bryan Pardo

Signal Processing · · cited 14× · DOI ↗ · more like this ·

While programming an audio synthesizer can be difficult, if a user has a general idea of the sound they are trying to program, they may be able to imitate it with their voice. In this technical demonstration, we demonstrate SynthAssist, a system that allows the user to program an audio synthesizer using vocal imitation and interactive feedback. This system treats synthesizer programming as an audio information retrieval task. To account for the limitations of the human voice, it compares vocal imitations to synthesizer sounds by using both absolute and relative temporal shapes of relevant audio features, and it refines the query and feature weights using relevance feedback.

Synthassist: Querying An Audio Synthesizer By Vocal Imitation

Mark Cartwright, Bryan Pardo

Signal Processing · Zenodo (CERN European Organization for Nuclear Research) · · cited 13× · DOI ↗ · more like this ·

Programming an audio synthesizer can be a difficult task for many. However, if a user has a general idea of the sound they are trying to program, they may be able to imitate it with their voice. This paper presents SynthAssist, a system for interactively searching the synthesis space of an audio synthesizer. In this work, we present how to use the system for querying a database of audio synthesizer patches (i.e. settings/parameters) by vocal imitation and user feedback. To account for the limitations of the human voice, it uses both absolute and relative time series representations of features and relevance feedback on both the feature weights and time series to refine the query. The method presented in this paper can be used to search through large databases of previously existing ``factory presets" or program a synthesizer using the data-driven approach to automatic synthesizer…

THE NUMBER OF CONJUGACY CLASSES OF CERTAIN FINITE GROUPS

Mark Cartwright

Discrete Mathematics and Combinatorics · The Quarterly Journal of Mathematics · · cited 11× · DOI ↗ · more like this ·

Journal Article THE NUMBER OF CONJUGACY CLASSES OF CERTAIN FINITE GROUPS Get access MARK CARTWRIGHT MARK CARTWRIGHT Christ ChurchOxford OX11DP Search for other works by this author on: Oxford Academic Google Scholar The Quarterly Journal of Mathematics, Volume 36, Issue 4, December 1985, Pages 393–404, https://doi.org/10.1093/qmath/36.4.393 Published: 01 December 1985 Article history Received: 12 March 1985 Published: 01 December 1985

Eliciting Confidence for Improving Crowdsourced Audio Annotations

Ana Elisa Méndez Méndez, Mark Cartwright, Juan Pablo Bello +1 more

Computer Science Applications · Proceedings of the ACM on Human-Computer Interaction · · cited 10× · DOI ↗ · more like this ·

In this work we explore confidence elicitation methods for crowdsourcing "soft" labels, e.g., probability estimates, to reduce the annotation costs for domains with ambiguous data. Machine learning research has shown that such "soft" labels are more informative and can reduce the data requirements when training supervised machine learning models. By reducing the number of required labels, we can reduce the costs of slow annotation processes such as audio annotation. In our experiments we evaluated three confidence elicitation methods: 1) "No Confidence" elicitation, 2) "Simple Confidence" elicitation, and 3) "Betting" mechanism for confidence elicitation, at both individual (i.e., per participant) and aggregate (i.e., crowd) levels. In addition, we evaluated the interaction between confidence elicitation methods, annotation types (binary, probability, and z-score derived probability)…

Machine-Crowd-Expert Model for Increasing User Engagement and Annotation Quality

Ana Elisa Méndez Méndez, Mark Cartwright, Juan Pablo Bello

Computer Science Applications · · cited 10× · DOI ↗ · more like this ·

Crowdsourcing and active learning have been combined in the past with the goal of reducing annotation costs. However, two issues arise with using AL and crowdsourcing: quality of the labels and user engagement. In this work, we propose a novel machine -> crowd <- expert loop model where the forward connections of the loop aim to improve the quality of the labels and the backward connections aim to increase user engagement. In addition, we propose a research agenda for evaluating the model.

Few-Shot Drum Transcription in Polyphonic Music

Yu Wang, Justin Salamon, Mark Cartwright +2 more

cs.SD · · cited 9× · arXiv ↗ · more like this ·

Data-driven approaches to automatic drum transcription (ADT) are often limited to a predefined, small vocabulary of percussion instrument classes. Such models cannot recognize out-of-vocabulary classes nor are they able to adapt to finer-grained vocabularies. In this work, we address open vocabulary ADT by introducing few-shot learning to the task. We train a Prototypical Network on a synthetic dataset and evaluate the model on multiple real-world ADT datasets with polyphonic accompaniment. We show that, given just a handful of selected examples at inference time, we can match and in some cases outperform a state-of-the-art supervised ADT approach under a fixed vocabulary setting. At the same time, we show that our model can successfully generalize to finer-grained or extended vocabularies unseen during training, a scenario where supervised approaches cannot operate at all. We provide a…

Crowdsourced Pairwise-Comparison for Source Separation Evaluation

Mark Cartwright, Bryan Pardo, Gautham J. Mysore

Signal Processing · · cited 9× · DOI ↗ · more like this ·

Automated objective methods of audio source separation evaluation are fast, cheap, and require little effort by the investigator. However, their output often correlates poorly with human quality assessments and typically require ground-truth (perfectly separated) signals to evaluate algorithm performance. Subjective multi-stimulus human ratings (e.g. MUSHRA) of audio quality are the gold standard for many tasks, but they are slow and require a great deal of effort to recruit participants and run listening tests. Recent work has shown that a crowdsourced multi-stimulus listening test can have results comparable to lab-based multi-stimulus tests. While these results are encouraging, MUSHRA multi-stimulus tests are limited to evaluating 12 or fewer stimuli, and they require ground-truth stimuli for reference. In this work, we evaluate a web-based pairwise-comparison listening approach that…

A Study on Robustness to Perturbations for Representations of Environmental Sound

Sangeeta Srivastava, Ho-Hsiang Wu, Joao Rulff +5 more

cs.SD · · cited 7× · arXiv ↗ · more like this ·

Audio applications involving environmental sound analysis increasingly use general-purpose audio representations, also known as embeddings, for transfer learning. Recently, Holistic Evaluation of Audio Representations (HEAR) evaluated twenty-nine embedding models on nineteen diverse tasks. However, the evaluation's effectiveness depends on the variation already captured within a given dataset. Therefore, for a given data domain, it is unclear how the representations would be affected by the variations caused by myriad microphones' range and acoustic conditions -- commonly known as channel effects. We aim to extend HEAR to evaluate invariance to channel effects in this work. To accomplish this, we imitate channel effects by injecting perturbations to the audio signal and measure the shift in the new (perturbed) embeddings with three distance measures, making the evaluation…