arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

New this week

New in cs.MM Multimedia

17 papers in cs.MM from 6 November to 12 November 2026.

Includes papers cross-listed here; the counts on the field chips are by primary category.

More fields

Whole archives

showing 1–17 of 17, newest first RSS

Augmented Hybrid TV Services. Habits, Preferences, Opinions and Expectations of Spanish Consumers

Fernando Boronat, Bernat Roig, Enric Sigalat +1 more

cs.MM · · cited 0× · arXiv ↗ · more like this ·

The convergence of hybrid broadcast and broadband delivery, companion devices, and immersive technologies enables new forms of interactive, personalized, and accessible TV services. This paper introduces the concept of Augmented Hybrid TV services and presents an exploratory user requirements and preliminary acceptance assessment for 5 representative use cases. They were selected through an exploratory stakeholder consultation and 3 co-design workshops with audiovisual professionals and participants from organizations supporting people with visual or hearing impairments. They combine conventional TV with complementary content delivered to connected devices, including AR/VR HMD. They were subsequently evaluated through a cross-sectional CAWI survey of 626 Internet and connected TV users in Spain. Each participant viewed standardized video demonstrations of 2 randomly assigned cases and…

RADIANT: Redundancy-Aware Information Allocation for Universal Multi-Modal Fusion

Shenghai Wu, Yue Lin

cs.CV · · cited 0× · arXiv ↗ · more like this ·

Universal multi-modal fusion often fails not because modalities are insufficiently aligned, but because the fusion process allocates information poorly: redundant evidence is repeatedly amplified, modality-unique evidence is over-smoothed during alignment, and task-dependent synergistic evidence is insufficiently activated. We present RADIANT (Redundancy-Aware Dynamic Information Allocation Network), a task-general framework that organizes fusion around three complementary roles: suppressing redundant cross-modal overlap, preserving modality-unique evidence, and composing task-conditioned synergy through cross-modal interaction. Specifically, stable matrix-exponential projections are used to reduce interference among shared and modality-unique streams, while task memory with hard positive updates selects which synergistic interactions should be activated for the current fusion regime.…

LUMIR: Decoupled Multimodal Representation Learning from Heterogeneous Data for Next-POI Recommendation

Zehui Wang, Wolfram Höpken, Dietmar Jannach

cs.IR · · cited 0× · arXiv ↗ · more like this ·

As multimodal tourism-related information collected from heterogeneous data sources has become increasingly available, learning effective representations has attracted attention for improving recommendation performance. Existing approaches often tightly couple multimodal fusion with the architecture and optimization objective of a specific downstream model, which limits the generality and reusability of the learned representations. To address these limitations, we propose LUMIR, a decoupled multimodal representation learning framework based on contrastive learning. Multimodal information collected from heterogeneous data sources is first semantically unified into a shared textual representation space using large language models (LLMs). An interaction-aware fusion mechanism is then designed to model cross-modal interactions, while sequential signals are incorporated as a structural…

Data Quality Challenges in Multimodal Tourism Recommender Systems

Zehui Wang, Wolfram Höpken, Dietmar Jannach

cs.IR · · cited 0× · arXiv ↗ · more like this ·

Modern recommender systems increasingly consider diverse data modalities to enhance model performance. However, the quality of the underlying multimodal data (e.g. data from social media platforms) has received limited attention so far. Focusing on tourism recommendation, we examine real-world challenges encountered when using datasets from Yelp and Instagram. Issues include noise from fake reviews, hallucinated content from LLMs, modality contradictions, and demographic bias. We outline future directions such as quality-aware fusion, structured and controlled LLM-based content generation to reduce hallucinations, and user group differentiation to improve the robustness and reliability of multimodal recommendation systems.

Joint Activation and Weight Quantization for Learned Image Compression

Yile Tu, Heming Sun

eess.IV · · cited 0× · arXiv ↗ · more like this ·

Learned image compression (LIC) models achieve competitive rate-distortion performance with increasingly large networks, making network quantization important for efficient deployment. Directly quantizing LIC models remains difficult in post-training quantization (PTQ) because compression performance is highly sensitive to quantization noise. Existing methods typically focus on optimizing either the weights or the activations, resulting in substantial rate-distortion degradation. To address this limitation, we introduce a two-stage pipeline that uses SmoothQuant to balance activation and weight ranges, followed by GPTQ to adapt the transformed weights. The proposed joint search framework optimizes the balancing transformation and adapted weights under the rate-distortion objective, accounting for the coupling between the two stages. Experiments across multiple LIC architectures and…

DASWeave: Weaving Evidence across Speaking Conditions for Multimodal Depression, Anxiety, and Stress Screening

Binyan Chen, Tong Pan, Yadong Liu +2 more

cs.MM · · cited 0× · arXiv ↗ · more like this ·

Multimodal screening for depression, anxiety, and stress in children and adolescents uses verbal and nonverbal behaviour. Most existing multimodal methods learn from a single recording or interview. Yet a participant's behaviour changes with the speaking condition, and these changes hold evidence that is invisible in a single recording. We propose DASWeave to bring this evidence into a neural network that encodes each response and then combines all responses of a participant. We train supervised trees to find which behavioural statistics matter for screening, both within each response and in how they change across conditions. We feed the tree outputs into the network both before and after this combination. We fine-tune a large language model to relate what a participant says to how they say it across conditions, and feed its representation into the network after the combination. The…

ABRR: Transforming Temporal Sentence Grounding with Wasserstein-Adaptive Refinement and Precision Ranking

Xiang Fang, Wanlong Fang, Junhao Dong +1 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

Temporal Sentence Grounding (TSG) aims to localize temporal segments in untrimmed videos that correspond to natural language queries, a critical task for video understanding with applications in video retrieval, summarization, and human-computer interaction. Despite significant progress driven by transformer-based models, existing TSG methods suffer from two key limitations: quantization errors due to temporal downsampling, which misalign segment boundaries, and suboptimal ranking of predictions based on semantic matching rather than localization precision. To address these challenges, we propose the Adaptive Boundary Refinement and Ranking (ABRR) framework, a novel, model-agnostic post-processing approach that enhances the performance of any TSG model without retraining. ABRR integrates three innovative components: (1) Temporal Distribution Smoothing, which mitigates noise in predicted…

Multimodal Judges Flag, Humans Decide: A Review-and-Edit Framework for AI-Generated Product Video Ads

Anoubhav Agarwaal, Sukesh Perla, Karthikeyan Mohanraj +1 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

Generative pipelines can now turn every product page of an e-commerce catalog into a short video ad, but deciding whether such an ad is usable is not one good-or-bad call. We describe how we operationalized the review and editing of these ads so that both can keep pace with generation: reviewers go through each ad scene by scene and answer six questions, on the product, scene quality, pronunciation, script, sound and captions; judges built from vision-language, speech and language models file each finding under the matching question, and reviewers accept or reject each finding and add what the judges missed. Each confirmed issue then opens an edit tool matched to it. The questions and the routing from a finding to its fix do not depend on the models that generate the ad. Across 120 ads from our pipeline, reviewed with judge findings on screen, reviewers marked the product wrong on 60%…

Artifact Filtering versus Artifact-Compensated Training in JPEG to JPEG AI Recompression

Igor Podtsepko, Yuriy Kim, Karam Mahfod +4 more

eess.IV · · cited 0× · arXiv ↗ · more like this ·

In this paper, we consider the problem of recompression of images from JPEG to JPEG AI format. We show that direct recompression from JPEG to JPEG~AI trained on uncompressed images is accompanied by the preservation of the JPEG image artifacts in the resulting JPEG~AI image. In order to avoid it, we evaluate an approach where an artifacts removal neural network is first applied to the input JPEG image as an external pre-filter. However, in this case, the processing time is increased by 29 times. Therefore, we propose an artifact-compensated training for JPEG AI and show that it is able to combine pre-filtering against target distortions with efficient compression without increase in the processing time.

BARS: Beat-Adaptive Rap Synthesis via Flow Matching

Taemin Cho, Hector Martel, Enyan Koh

cs.SD · · cited 0× · arXiv ↗ · more like this ·

We present BARS (Beat-Adaptive Rap Synthesis), a non-autoregressive rapping voice generation model based on Flow Matching, conditioned on a beat track (instrumental backing), lyrics, and a short reference voice for zero-shot timbre control. Unlike prior autoregressive rap synthesis, where rap pace and output duration are strictly determined by the beat, BARS enables independent control over the rap tempo, allowing flexible-length generation without explicit duration prediction or phoneme-level alignment. We encode lyrics using a pretrained ByT5 encoder with a lightweight Transformer adapter trained jointly with the model, without relying on speech-pretrained text encoders, enabling natural handling of non-standard orthography (e.g., slang, abbreviations) and numerals common in rap lyrics. To address alignment instabilities, such as token skipping or repetition, under strong beat…

YTShorts-92K: An Interpretable Multimodal Dataset of YouTube Shorts

Celian Charrin, Sidonie Abuzakuk, Artúr Vlaszov +2 more

cs.MM · · cited 0× · arXiv ↗ · more like this ·

Short-form video has become one of the dominant formats of online content, yet large-scale datasets enabling the analysis of what makes these videos successful remain limited. Existing resources often focus on post-publication popularity signals or rely on non-interpretable representations such as high-dimensional embeddings. Here, we introduce YTShorts-92K, a dataset of 92,654 YouTube Shorts collected from 345 creators and spanning 12 manually defined content categories. Each video is enriched with an interpretable multimodal feature set covering metadata, textual information, visual characteristics from frames and thumbnails, and cross-modal consistency measures. We describe the collection methodology, data cleaning process, and feature extraction pipeline, followed by descriptive analyses of the diversity of creators, topics, and extracted features. This dataset offers a reusable…

TIB: Sample-wise Tempered Information Bottleneck for Multimodal Attribution beyond Alignment Assumption

Qiming Huang, Peixi Liu, Jianbo Jiao

cs.CV · · cited 0× · arXiv ↗ · more like this ·

Multimodal (e.g. vision-language, in this work) attribution aims to interpret the models without access to ground-truth explanatory supervision. Existing multimodal attribution methods typically rely on paired modalities as proxy supervision, implicitly assuming that image-text pairs are semantically aligned. Information bottleneck (IB)-based attribution methods follow this general paradigm and instantiate it through a global sufficiency-compression objective. In realistic multimodal datasets, however, this assumption is frequently violated due to partial semantic misalignment and unreliable cross-modal correspondences. Through theoretical analysis, we show that, under such misalignment, information-bottleneck-based attribution objectives exhibit a structural limitation: by assuming equal reliability across all proxy supervision signals, they induce posterior over-contraction and forced…

When Coding Agents Get Arms: Safety Risks Across the Digital-Physical Boundary

Yiming Sun, Mi Zhang

cs.RO · · cited 0× · arXiv ↗ · more like this ·

Advances in spatial understanding have enabled frontier models to power coding agents that control a real robot arm zero-shot and end-to-end, without task-specific demonstrations or retraining. This brings two new safety challenges. First, the extensive knowledge of frontier models lets these agents perform tasks that earlier VLA and WAM paradigms could not, opening the door to new forms of capability misuse. Second, coding agents can access digital resources such as web pages and emails. Threats in these resources, such as indirect prompt injection, can carry over into the physical world when the agent controls a robot. We therefore introduce ArmedAgent, the first systematic benchmark to evaluate both risks for coding agents performing zero-shot robot control. It covers robustness, malicious use, and indirect injection across 36 tasks in 5 scenarios, evaluated in simulation and…

The RenAIssance: Caption-Guided Diffusion Models for Historical-Art Visual Media

Zhanyu Tuo

cs.CV · · cited 0× · arXiv ↗ · more like this ·

Generative AI can expand cultural collections into prompt-responsive visual media, but historical artworks rarely provide the paired descriptions needed to adapt modern image generators. We present The RenAIssance, a scalable pipeline that turns a collection of historical European artworks into a reusable text-to-image media resource. The pipeline generates scene-level captions with BLIP and uses the resulting approximately 15,000 image--text pairs from DEArt to train a compact LoRA adapter for Stable Diffusion v1.5. The adapter shifts generated images toward the historical-art distribution while retaining the base model's ability to render novel concepts. On historical and contemporary prompts of different lengths, it improves FID to the DEArt reference distribution in every setting with only a modest decrease in CLIP text--image similarity. Qualitative results show that the same…

SPDMX: Large-Scale Open Stem Audio From Symbolic Music

Phillip Long, Eduardo Escoto, Julian McAuley +1 more

cs.SD · · cited 0× · arXiv ↗ · more like this ·

Large public datasets have driven recent progress in machine learning, from text and vision to speech and general audio. Music remains an outlier, as copyright blocks redistribution of training audio, while many modern tasks need individual stems rather than mixtures. Existing multitrack resources trade off realism, scale, and licensing, while synthetic alternatives rely on proprietary VSTs or naive open rendering that yields weak stems. We introduce SPDMX, a listening-guided rendering of public-domain PDMX scores into an openly-licensed stem corpus with aligned scores and an open render pipeline: $253{,}982$ songs, $490{,}159$ mono stems, and $5{,}469$ song-hours at $44.1$ kHz under CC BY 4.0, with paired MIDI and identifiers linking to PDMX metadata. $88{,}403$ songs ($34.8\%$; $2{,}734$ song-hours) provide multitrack supervision. To our knowledge, this is the largest openly-licensed…

UniMore: Fine-Grained Supervision for Unified Understanding and Generation

Haoyue Bai, Yinyi Luo, Yicheng He +3 more

cs.MM · · cited 0× · arXiv ↗ · more like this ·

Unified multimodal models combine image understanding and generation within one system, creating an opportunity to improve both capabilities through paired image--text supervision. We examine the outputs of a pretrained unified model in both directions and observe a loss of specificity: generated images can contain the requested objects with incorrect attributes, while captions identify a scene without expressing its distinguishing details. These failures call for supervision that specifies both the visual content to realize and the descriptive differences to distinguish. We propose UniMore, a post-training method that derives these complementary learning tasks from the fine-grained content of image--text pairs. The generation route predicts grounded crops from their corresponding phrases, matching the scope of the condition and visual target while using the native visual objective. The…

Refer-CityFlow and M3Track: A Benchmark Dataset and Reference Framework for Language-Driven Multi-Target Multi-Camera Tracking

Phuong T. Nguyen, Hao Vo, Tam Bang +8 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

Referring multi-object tracking enables users to specify targets through natural language, but existing Refer-MOT benchmarks primarily focus on single-camera settings. We formalize Referring Multi-Target Multi-Camera Tracking (Refer-MTMCT), where free-form queries specify identity-consistent trajectories to be recovered across a network of sequential, sparse-overlap cameras. We introduce Refer-CityFlow, a benchmark built on CityFlow's global cross-camera identity annotations and augmented with language queries requiring joint language grounding, cross-camera association, and spatio-temporal reasoning. Refer-CityFlow covers 96 videos from 40 roadside cameras with 660+ identity-consistent cross-camera trajectories, 4,362 trajectory-level descriptions, and 7,699 queries, including 1,699 real-user queries collected from 50 participants. Reference experiments on the four-camera Scenario 02…