arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Computer Science · Signal Processing

An Investigation Of Rotary Position Embedding For Speech Enhancement

Moran Chen, Mingjiang Wang

See on the map

Abstract

Transformer architecture has driven significant advancements speech enhancement (SE). Due to the position-agnostic characteristic of Transformers, positional embedding has become a crucial component to make Transformers to distinguish the order of elements among the input sequence. Rotary Position Embedding (RoPE) demonstrates exceptional performance on language models, particularly in processing the variable-length input sequences. However, the impact of RoPE on SE has not been thoroughly investigated. To address this, this paper introduces RoPE into SE and proposes the Transformer-RoP architecture, then comprehensively evaluates its effective-ness. Experiments were conducted with a standard Transformer architecture as the baseline and PSM as the training target. The results illustrate that Transformer-RoP achieves substantial improvements across all metrics.

More like this