Author

Christopher Simic

4 papers · 2023–2025

2023peak 2 in 20232025

Frequent co-authors

Papers

showing 1–4 of 4 RSS

Shared Multi-modal Embedding Space for Face-Voice Association

Christopher Simic, Korbinian Riedhammer, Tobias Bocklet

cs.SD · 4 Dec 2025 · cited 0× · arXiv ↗ · more like this ·

The FAME 2026 challenge comprises two demanding tasks: training face-voice associations combined with a multilingual setting that includes testing on languages on which the model was not trained. Our approach consists of separate uni-modal processing pipelines with general face and voice feature extraction, complemented by additional age-gender feature extraction to support prediction. The resulting single-modal features are projected into a shared embedding space and trained with an Adaptive Angular Margin (AAM) loss. Our approach achieved first place in the FAME 2026 challenge, with an average Equal-Error Rate (EER) of 23.99%.

Self-Supervised Adaptive AV Fusion Module for Pre-Trained ASR Models

Christopher Simic, Tobias Bocklet

cs.SD · 21 Dec 2023 · cited 0× · arXiv ↗ · more like this ·

Automatic speech recognition (ASR) has reached a level of accuracy in recent years, that even outperforms humans in transcribing speech to text. Nevertheless, all current ASR approaches show a certain weakness against ambient noise. To reduce this weakness, audio-visual speech recognition (AVSR) approaches additionally consider visual information from lip movements for transcription. This additional modality increases the computational cost for training models from scratch. We propose an approach, that builds on a pre-trained ASR model and extends it with an adaptive upstream module, that fuses audio and visual information. Since we do not need to train the transformer structure from scratch, our approach requires a fraction of the computational resources compared to traditional AVSR models. Compared to current SOTA systems like AV-HuBERT, our approach achieves an average improvement of…

The Hochschul-Assistenz-System

Thomas Ranzenberger, Tobias Bocklet, Steffen Freisinger +9 more

Computer Science Applications · OHMdok (Technische Hochschule Nürnberg) · 5 Mar 2023 · cited 0× · OpenAlex ↗ · more like this ·

The usage of e-learning platforms, online lectures and online meetings for academic teaching increased during the Covid-19 pandemic. Lecturers created video lectures, screencasts, or audio podcasts for online learning. The Hochschul-Assistenz-System (HAnS) is a learning experience platform that uses machine learning (ML) methods to support students and lecturers in the online learning and teaching processes. HAnS is being developed in multiple iterations as an agile open-source collaborative project supported by multiple universities and partners. This paper presents the current state of the development of HAnS on German video lectures.

Adapter-Based Multi-Agent AVSR Extension for Pre-Trained ASR Models

Christopher Simic, Korbinian Riedhammer, Tobias Bocklet

cs.SD · 3 Feb 2025 · arXiv ↗ · more like this ·

We present an approach to Audio-Visual Speech Recognition that builds on a pre-trained Whisper model. To infuse visual information into this audio-only model, we extend it with an AV fusion module and LoRa adapters, one of the most up-to-date adapter approaches. One advantage of adapter-based approaches, is that only a relatively small number of parameters are trained, while the basic model remains unchanged. Common AVSR approaches train single models to handle several noise categories and noise levels simultaneously. Taking advantage of the lightweight nature of adapter approaches, we train noise-scenario-specific adapter-sets, each covering individual noise-categories or a specific noise-level range. The most suitable adapter-set is selected by previously classifying the noise-scenario. This enables our models to achieve an optimum coverage across different noise-categories and…