arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Robin Scheibler

57 papers · 2010–2026

2010: 12011: 12012: 02013: 12014: 12015: 12016: 12017: 42018: 32019: 52020: 112021: 62022: 62023: 32024: 32025: 92026: 1
2010peak 11 in 20202026

Frequent co-authors

Papers

showing 1–25 of 57 RSS

Pyroomacoustics: A Python package for audio room simulations and array processing algorithms

Robin Scheibler, Eric Bezzam, Ivan Dokmanić

cs.SD · · cited 731× · arXiv ↗ · more like this ·

We present pyroomacoustics, a software package aimed at the rapid development and testing of audio array processing algorithms. The content of the package can be divided into three main components: an intuitive Python object-oriented interface to quickly construct different simulation scenarios involving multiple sound sources and microphones in 2D and 3D rooms; a fast C implementation of the image source model for general polyhedral rooms to efficiently generate room impulse responses and simulate the propagation between sources and receivers; and finally, reference implementations of popular algorithms for beamforming, direction finding, and adaptive filtering. Together, they form a package with the potential to speed up the time to market of new algorithms by significantly reducing the implementation overhead in the performance evaluation step.

Diffusion-based Generative Speech Source Separation

Robin Scheibler, Youna Ji, Soo-Whan Chung +3 more

eess.AS · · cited 76× · arXiv ↗ · more like this ·

We propose DiffSep, a new single channel source separation method based on score-matching of a stochastic differential equation (SDE). We craft a tailored continuous time diffusion-mixing process starting from the separated sources and converging to a Gaussian distribution centered on their mixture. This formulation lets us apply the machinery of score-based generative modelling. First, we train a neural network to approximate the score function of the marginal probabilities or the diffusion-mixing process. Then, we use it to solve the reverse time SDE that progressively separates the sources starting from their mixture. We propose a modified training strategy to handle model mismatch and source permutation ambiguity. Experiments on the WSJ0 2mix dataset demonstrate the potential of the method. Furthermore, the method is also suitable for speech enhancement and shows performance…

Independent Vector Analysis with more Microphones than Sources

Robin Scheibler, Nobutaka Ono

cs.SD · · cited 71× · arXiv ↗ · more like this ·

We extend frequency-domain blind source separation based on independent vector analysis to the case where there are more microphones than sources. The signal is modelled as non-Gaussian sources in a Gaussian background. The proposed algorithm is based on a parametrization of the demixing matrix decreasing the number of parameters to estimate. Furthermore, orthogonal constraints between the signal and background subspaces are imposed to regularize the separation. The problem can then be posed as a constrained likelihood maximization. We propose efficient alternating updates guaranteed to converge to a stationary point of the cost function. The performance of the algorithm is assessed on simulated signals. We find that the separation performance is on par with that of the conventional determined algorithm at a fraction of the computational cost.

Fast and Stable Blind Source Separation with Rank-1 Updates

Robin Scheibler, Nobutaka Ono

Signal Processing · IEEE Conference Proceedings · · cited 61× · DOI ↗ · more like this ·

We propose a new algorithm for the blind source separation of acoustic sources. This algorithm is an alternative to the popular auxiliary function based independent vector analysis using iterative projection (AuxIVA-IP). It optimizes the same cost function, but instead of alternate updates of the rows of the demixing matrix, we propose a sequence of rank-1 updates. Remarkably, and unlike the previous method, the resulting updates do not require matrix inversion. Moreover, their computational complexity is quadratic in the number of microphones, rather than cubic in AuxIVA-IP. In addition, we show that the new method can be derived as alternate updates of the steering vectors of sources. Accordingly, we name the method iterative source steering (AuxIVA-ISS). Finally, we confirm in simulated experiments that the proposed algorithm separates sources just as well as AuxIVA-IP, at a lower…

TorchAudio 2.1: Advancing speech recognition, self-supervised learning, and audio processing components for PyTorch

Jeff Hwang, Moto Hira, Caroline Chen +21 more

eess.AS · · cited 50× · arXiv ↗ · more like this ·

TorchAudio is an open-source audio and speech processing library built for PyTorch. It aims to accelerate the research and development of audio and speech technologies by providing well-designed, easy-to-use, and performant PyTorch components. Its contributors routinely engage with users to understand their needs and fulfill them by developing impactful features. Here, we survey TorchAudio's development principles and contents and highlight key features we include in its latest version (2.1): self-supervised learning pre-trained pipelines and training recipes, high-performance CTC decoders, speech recognition models and training recipes, advanced media I/O capabilities, and tools for performing forced alignment, multi-channel speech enhancement, and reference-less speech assessment. For a selection of these features, through empirical studies, we demonstrate their efficacy and show that…

Raking the Cocktail Party

Ivan Dokmanić, Robin Scheibler, Martin Vetterli

cs.SD · · cited 48× · arXiv ↗ · more like this ·

We present the concept of an acoustic rake receiver---a microphone beamformer that uses echoes to improve the noise and interference suppression. The rake idea is well-known in wireless communications; it involves constructively combining different multipath components that arrive at the receiver antennas. Unlike spread-spectrum signals used in wireless communications, speech signals are not orthogonal to their shifts. Therefore, we focus on the spatial structure, rather than temporal. Instead of explicitly estimating the channel, we create correspondences between early echoes in time and image sources in space. These multiple sources of the desired and the interfering signal offer additional spatial diversity that we can exploit in the beamformer design. We present several "intuitive" and optimal formulations of acoustic rake receivers, and show theoretically and numerically that the…

A Fast Hadamard Transform for Signals with Sub-linear Sparsity in the Transform Domain

Robin Scheibler, Saeid Haghighatshoar, Martin Vetterli

cs.IT · · cited 47× · arXiv ↗ · more like this ·

A new iterative low complexity algorithm has been presented for computing the Walsh-Hadamard transform (WHT) of an $N$ dimensional signal with a $K$-sparse WHT, where $N$ is a power of two and $K = O(N^α)$, scales sub-linearly in $N$ for some $0 < α< 1$. Assuming a random support model for the non-zero transform domain components, the algorithm reconstructs the WHT of the signal with a sample complexity $O(K \log_2(\frac{N}{K}))$, a computational complexity $O(K\log_2(K)\log_2(\frac{N}{K}))$ and with a very high probability asymptotically tending to 1. The approach is based on the subsampling (aliasing) property of the WHT, where by a carefully designed subsampling of the time domain signal, one can induce a suitable aliasing pattern in the transform domain. By treating the aliasing patterns as parity-check constraints and borrowing ideas from erasure correcting sparse-graph codes, the…

ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understanding

Yen-Ju Lu, Xuankai Chang, Chenda Li +10 more

eess.AS · · cited 37× · arXiv ↗ · more like this ·

This paper presents recent progress on integrating speech separation and enhancement (SSE) into the ESPnet toolkit. Compared with the previous ESPnet-SE work, numerous features have been added, including recent state-of-the-art speech enhancement models with their respective training and evaluation recipes. Importantly, a new interface has been designed to flexibly combine speech enhancement front-ends with other tasks, including automatic speech recognition (ASR), speech translation (ST), and spoken language understanding (SLU). To showcase such integration, we performed experiments on carefully designed synthetic datasets for noisy-reverberant multi-channel ST and SLU tasks, which can be used as benchmark corpora for future research. In addition to these new tasks, we also use CHiME-4 and WSJ0-2Mix to benchmark multi- and single-channel SE approaches. Results show that the integration…

FRIDA: FRI-Based DOA Estimation for Arbitrary Array Layouts

Hanjie Pan, Robin Scheibler, Eric Bezzam +2 more

cs.SD · · cited 35× · arXiv ↗ · more like this ·

In this paper we present FRIDA---an algorithm for estimating directions of arrival of multiple wideband sound sources. FRIDA combines multi-band information coherently and achieves state-of-the-art resolution at extremely low signal-to-noise ratios. It works for arbitrary array layouts, but unlike the various steered response power and subspace methods, it does not require a grid search. FRIDA leverages recent advances in sampling signals with a finite rate of innovation. It is based on the insight that for any array layout, the entries of the spatial covariance matrix can be linearly transformed into a uniformly sampled sum of sinusoids.

Surrogate Source Model Learning for Determined Source Separation

Robin Scheibler, Masahito Togami

eess.AS · · cited 32× · arXiv ↗ · more like this ·

We propose to learn surrogate functions of universal speech priors for determined blind speech separation. Deep speech priors are highly desirable due to their high modelling power, but are not compatible with state-of-the-art independent vector analysis based on majorization-minimization (AuxIVA), since deriving the required surrogate function is not easy, nor always possible. Instead, we do away with exact majorization and directly approximate the surrogate. Taking advantage of iterative source steering (ISS) updates, we back propagate the permutation invariant separation loss through multiple iterations of AuxIVA. ISS lends itself well to this task due to its lower complexity and lack of matrix inversion. Experiments show large improvements in terms of scale invariant signal-to-distortion (SDR) ratio and word error rate compared to baseline methods. Training is done on two speakers…

SDR -- Medium Rare with Fast Computations

Robin Scheibler

eess.AS · · cited 27× · arXiv ↗ · more like this ·

We revisit the widely used bss eval metrics for source separation with an eye out for performance. We propose a fast algorithm fixing shortcomings of publicly available implementations. First, we show that the metrics are fully specified by the squared cosine of just two angles between estimate and reference subspaces. Second, large linear systems are involved. However, they are structured, and we apply a fast iterative method based on conjugate gradient descent. The complexity of this step is thus reduced by a factor quadratic in the distortion filter size used in bss eval, usually 512. In experiments, we assess speed and numerical accuracy. Not only is the loss of accuracy due to the approximate solver acceptable for most applications, but the speed-up is up to two orders of magnitude in some, not so extreme, cases. We confirm that our implementation can train neural networks, and…

Fast Independent Vector Extraction by Iterative SINR Maximization

Robin Scheibler, Nobutaka Ono

cs.SD · · cited 22× · arXiv ↗ · more like this ·

We propose fast independent vector extraction (FIVE), a new algorithm that blindly extracts a single non-Gaussian source from a Gaussian background. The algorithm iteratively computes beamforming weights maximizing the signal-to-interference-and-noise ratio for an approximate noise covariance matrix. We demonstrate that this procedure minimizes the negative log-likelihood of the input data according to a well-defined probabilistic model. The minimization is carried out via the auxiliary function technique whereas, unlike related methods, the auxiliary function is globally minimized at every iteration. Numerical experiments are carried out to assess the performance of FIVE. We find that it is vastly superior to competing methods in terms of convergence speed, and has high potential for real-time applications.

Independent Vector Analysis via Log-Quadratically Penalized Quadratic Minimization

Robin Scheibler

eess.SP · · cited 17× · arXiv ↗ · more like this ·

We propose a new algorithm for blind source separation (BSS) using independent vector analysis (IVA). This is an improvement over the popular auxiliary function based IVA (AuxIVA) with iterative projection (IP) or iterative source steering (ISS). We introduce iterative projection with adjustment (IPA), where we update one demixing filter and jointly adjust all the other sources along its current direction. Each update involves solving a non-convex minimization problem that we term log-quadratically penalized quadratic minimization (LQPQM), that we think is of interest beyond this work. In the general case, we show that its global minimum corresponds to the largest root of a univariate function, reminiscent of modified eigenvalue problems. We propose a simple procedure based on Newton-Raphson to efficiently compute it. Numerical experiments demonstrate the effectiveness of the proposed…

Neural Diarization with Non-autoregressive Intermediate Attractors

Yusuke Fujita, Tatsuya Komatsu, Robin Scheibler +2 more

eess.AS · · cited 15× · arXiv ↗ · more like this ·

End-to-end neural diarization (EEND) with encoder-decoder-based attractors (EDA) is a promising method to handle the whole speaker diarization problem simultaneously with a single neural network. While the EEND model can produce all frame-level speaker labels simultaneously, it disregards output label dependency. In this work, we propose a novel EEND model that introduces the label dependency between frames. The proposed method generates non-autoregressive intermediate attractors to produce speaker labels at the lower layers and conditions the subsequent layers with these labels. While the proposed model works in a non-autoregressive manner, the speaker labels are refined by referring to the whole sequence of intermediate labels. The experiments with the two-speaker CALLHOME dataset show that the intermediate labels with the proposed non-autoregressive intermediate attractors boost the…

MM Algorithms for Joint Independent Subspace Analysis with Application to Blind Single and Multi-Source Extraction

Robin Scheibler, Nobutaka Ono

eess.SP · · cited 15× · arXiv ↗ · more like this ·

In this work, we propose efficient algorithms for joint independent subspace analysis (JISA), an extension of independent component analysis that deals with parallel mixtures, where not all the components are independent. We derive an algorithmic framework for JISA based on the majorization-minimization (MM) optimization technique (JISA-MM). We use a well-known inequality for super-Gaussian sources to derive a surrogate function of the negative log-likelihood of the observed data. The minimization of this surrogate function leads to a variant of the hybrid exact-approximate diagonalization problem, but where multiple demixing vectors are grouped together. In the spirit of auxiliary function based independent vector analysis (AuxIVA), we propose several updates that can be applied alternately to one, or jointly to two, groups of demixing vectors. Recently, blind extraction of one or more…

Joint Dereverberation and Separation with Iterative Source Steering

Taishi Nakashima, Robin Scheibler, Masahito Togami +1 more

eess.AS · · cited 14× · arXiv ↗ · more like this ·

We propose a new algorithm for joint dereverberation and blind source separation (DR-BSS). Our work builds upon the IRLMA-T framework that applies a unified filter combining dereverberation and separation. One drawback of this framework is that it requires several matrix inversions, an operation inherently costly and with potential stability issues. We leverage the recently introduced iterative source steering (ISS) updates to propose two algorithms mitigating this issue. Albeit derived from first principles, the first algorithm turns out to be a natural combination of weighted prediction error (WPE) dereverberation and ISS-based BSS, applied alternatingly. In this case, we manage to reduce the number of matrix inversion to only one per iteration and source. The second algorithm updates the ILRMA-T matrix using only sequential ISS updates requiring no matrix inversion at all. Its…

Computationally-Efficient Overdetermined Blind Source Separation Based on Iterative Source Steering

Yicheng Du, Robin Scheibler, Masahito Togami +2 more

Signal Processing · IEEE Signal Processing Letters · · cited 12× · DOI ↗ · more like this ·

This paper describesa computationally-efficient optimization algorithm for the blind source separation (BSS) of overdetermined mixtures. In the determined case, a matrix-inversion-free iterative source steering (ISS) algorithm has been proposed for estimating a square demixing matrix as a computationally-efficient alternative to the popular iterative projection (IP) algorithm. The IP algorithm is based on source-wise (i.e., row-wise) updates of the demixing matrix, and lends itself naturally to an extension to overdetermined independent vector analysis (IVA) called OverIVA. In contrast, the ISS algorithm changes the whole demixing matrix at every update, making its extension to the overdetermined case non-trivial. In this paper, we propose a modified ISS algorithm for OverIVA fully exploiting the computational savings of ISS. We also derive an overdetermined extension of independent…

Separake: Source Separation with a Little Help From Echoes

Robin Scheibler, Diego Di Carlo, Antoine Deleforge +1 more

cs.SD · · cited 12× · arXiv ↗ · more like this ·

It is commonly believed that multipath hurts various audio processing algorithms. At odds with this belief, we show that multipath in fact helps sound source separation, even with very simple propagation models. Unlike most existing methods, we neither ignore the room impulse responses, nor we attempt to estimate them fully. We rather assume that we know the positions of a few virtual microphones generated by echoes and we show how this gives us enough spatial diversity to get a performance boost over the anechoic case. We show improvements for two standard algorithms---one that uses only magnitudes of the transfer functions, and one that also uses the phases. Concretely, we show that multichannel non-negative matrix factorization aided with a small number of echoes beats the vanilla variant of the same algorithm, and that with magnitude information only, echoes enable separation where…

Blinkies: Sound-to-light conversion sensors and their application to speech enhancement and sound source localization

Robin Scheibler, Daiki Horiike, Nobutaka Ono

Signal Processing · · cited 10× · DOI ↗ · more like this ·

We introduce the use of blinkies for acoustic sensing and audio processing applications. Blinkies are low-power sensors with a microphone and an LED that can be easily distributed over a large area. The power of the LED is modulated by the sound intensity and the signals from all devices can be captured by a regular video camera. We present our design for blinkies and characterize the transmission function from blinky to captured video signal. The usefulness of such a system is demonstrated with two applications. First, we evaluate beamforming informed by a high-quality voice activity signal obtained from a blinky. Second, we investigate sound source localization using several blinkies distributed in a room.

Pyramic: Full Stack Open Microphone Array Architecture and Dataset

Robin Scheibler, Juan Azcarreta, René Beuchat +1 more

Signal Processing · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · · cited 10× · DOI ↗ · more like this ·

In this paper we introduce an open source and reproducible microphone array hardware design and an anechoic dataset recorded with this array. The Pyramic array has 48 microphones spread onto six identical modules connected to an FPGA-ARM combo. The arrangement of the six modules can be reconfigured to create a large number of geometries. We describe in detail the architecture of the array and make openly available all necessary hardware design files, VHDL code, and C libraries together with extensive documentation. This effectively enables replicability of part or all of the array. The curated dataset of anechoic measurements done using the Pyramic array comprises source locations with dense azimuth sampling at multiple heights, playing both test and speech signals. The manual calibration of source and microphone locations is assessed and improved upon using time-difference of arrival…

Raking echoes in the time domain

Robin Scheibler, Ivan Dokmanić, Martin Vetterli

Signal Processing · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · · cited 10× · DOI ↗ · more like this ·

The geometry of room acoustics is such that the reverberant signal can be seen as the same waveform emitted from multiple locations. In analogy with the rake receiver from wireless communications, we propose several beamforming strategies that exploit, rather than suppress, this additional spatio-temporal diversity. Unlike earlier work in the frequency domain, time domain designs allow to shape the impulse response of the beamformer. In particular, we can control perceptually relevant parameters, such as the amount of early echoes or the length of the beamformer response. Relying on the knowledge of the image sources positions, we derive different optimal beamformers. Leveraging perceptual cues, we show how to improve interference and noise reduction without degrading the perceptual quality. The designs are validated through simulation. Using early echoes is shown to strictly improve…

Blinkies: Open Source Sound-to-Light Conversion Sensors for Large-Scale Acoustic Sensing and Applications

Robin Scheibler, Nobutaka Ono

Signal Processing · IEEE Access · · cited 9× · DOI ↗ · more like this ·

We propose an open source hardware platform called Blinky, a sound-to-light conversion sensor that harvests sound power at low-rate and conveniently. Blinkies are made up of a central processing unit connected to two microphones and a few light-emitting devices, are powered by a battery, and protected by a robust enclosure. Distributed in space and combined with a conventional video camera, they allow to practically sense sound power over a very large area without hassle. We give a comprehensive overview of the proposed system and its potential applications. We describe the hardware design and trade-offs made. We provide a model for the channel between sound power measurements and signal acquired by the video camera. Because each sensor is potentially affected by a different attenuation due to the channel, we propose a calibration procedure to restore the scale of the measurements. The…

Effectiveness of Inter- and Intra-Subarray Spatial Features for Acoustic Scene Classification

Takao Kawamura, Yuma Kinoshita, Nobutaka Ono +1 more

Signal Processing · · cited 8× · DOI ↗ · more like this ·

In this paper, we investigate the effectiveness of spatial features for acoustic scene classification (ASC) with distributed microphones. Assuming that multiple subarrays, each containing multiple micro-phones, are distributed and synchronized, we consider two types of generalized cross-correlation phase transform (GCC-PHAT) as spatial features: the intra- and inter-subarray GCC-PHATs. They are obtained from channels within the same subarray and between different subarrays, respectively. The log-Mel spectrogram as a spectral feature and the intra- or inter-subarray GCC-PHAT are processed in the neural network. The experimental results show that increasing the number of channels did not markedly improve the ASC performance when using the spectral features alone. However, using either of the GCC-PHATs as the spatial feature together with the spectral features successfully improved the ASC…

Faster independent low-rank matrix analysis with pairwise updates of demixing vectors

Taishi Nakashima, Robin Scheibler, Yukoh Wakabayashi +1 more

Signal Processing · · cited 8× · DOI ↗ · more like this ·

In this paper, we present an algorithm for independent low-rank matrix analysis (ILRMA) of three or more sources that is faster than that for conventional ILRMA. In conventional ILRMA, demixing vectors are updated one by one by the iterative projection (IP) method. The update rules of IP are derived from a system of quadratic equations obtained by differentiating the objective function of ILRMA with respect to demixing vectors. This system of quadratic equations is called hybrid exact-approximate joint diagonalization (HEAD) and no closed-form solution is known yet for three or more sources. Recently, a method that can update two demixing vectors simultaneously has been proposed for independent vector analysis. The method is derived by reducing HEAD for two sources to a generalized eigenvalue problem and solving the problem. Furthermore, the pairwise updates have recently been extended…

Multi-modal Blind Source Separation with Microphones and Blinkies

Robin Scheibler, Nobutaka Ono

cs.SD · · cited 8× · arXiv ↗ · more like this ·

We propose a blind source separation algorithm that jointly exploits measurements by a conventional microphone array and an ad hoc array of low-rate sound power sensors called blinkies. While providing less information than microphones, blinkies circumvent some difficulties of microphone arrays in terms of manufacturing, synchronization, and deployment. The algorithm is derived from a joint probabilistic model of the microphone and sound power measurements. We assume the separated sources to follow a time-varying spherical Gaussian distribution, and the non-negative power measurement space-time matrix to have a low-rank structure. We show that alternating updates similar to those of independent vector analysis and Itakura-Saito non-negative matrix factorization decrease the negative log-likelihood of the joint distribution. The proposed algorithm is validated via numerical experiments.…