arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Keyu Fan

13 papers · 2023–2026

2023: 12024: 12025: 72026: 4
2023peak 7 in 20252026

Frequent co-authors

Papers

showing 1–13 of 13 RSS

<i>XStreamVGGT</i> : Extremely Memory‐Efficient Streaming Vision Geometry Grounded Transformer With KV Cache Compression

Zunhai Su, Weihao Ye, Hansen Feng +5 more

Computer Vision and Pattern Recognition · Journal of the Society for Information Display · · cited 0× · DOI ↗ · more like this ·

ABSTRACT Learning‐based 3D visual geometry models have significantly advanced with the advent of large‐scale transformers. Among these, StreamVGGT leverages frame‐wise causal attention to deliver robust and efficient streaming 3D reconstruction. However, it suffers from unbounded growth in the Key–Value (KV) cache due to the massive influx of vision tokens from multi‐image and long‐video inputs, leading to increased memory consumption and inference latency as input frames accumulate. To address this gap, we propose XStreamVGGT, a tuning‐free approach that seamlessly integrates pruning and quantization to systematically compress the KV cache, enabling extremely memory‐efficient streaming inference. Specifically, redundant KVs generated from multiframe inputs are first pruned to satisfy a fixed KV memory budget. This process employs an efficient token importance identification mechanism…

Hear-Your-Click: Interactive Object-Specific Video-to-Audio Generation

Yingshan Liang, Keyu Fan, Zhicheng Du +5 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

Video-to-audio (V2A) generation shows great potential in fields such as film production. Despite significant advances, current V2A methods relying on global video information struggle with complex scenes and generating audio tailored to specific objects. To address these limitations, we introduce Hear-Your-Click, an interactive V2A framework enabling users to generate sounds for specific objects by clicking on the frame. To achieve this, we propose Object-aware Contrastive Audio-Visual Fine-tuning (OCAV) with a Mask-guided Visual Encoder (MVE) to obtain object-level visual features aligned with audio. Furthermore, we tailor two data augmentation strategies, Random Video Stitching (RVS) and Mask-guided Loudness Modulation (MLM), to enhance the model's sensitivity to segmented objects. To measure audio-visual correspondence, we designed a new evaluation metric, the CAV score. Extensive…

TAGMO: Temporal Control Audio Generation for Multiple Visual Objects Without Training

Xinyu Zhang, Keyu Fan, Yiran Wang +5 more

Signal Processing · · cited 0× · DOI ↗ · more like this ·

With the great popularity of Sora, video-based audio generation has become indispensable. While numerous video-to-audio generation models have emerged, they frequently face difficulties including semantic incompatibilities and synchronization problems, especially in situations with multiple objects. To address these difficulties, we introduce TAGMO, a novel training-free audio generation method that offers precise time control for multi-object video scenarios. Our approach first employs object detection to obtain the class labels and temporal labels of each object, which are then structured and utilized as control conditions within a latent diffusion model (LDM) to generate multi-object audio. Additionally, we innovatively design a time mask based on the corresponding temporal labels and integrate it into the denoising process of the pre-trained audio generation model to achieve…

CushionCatch: A Compliant Catching Mechanism for Mobile Manipulators via Combined Optimization and Learning

Bingjie Chen, Keyu Fan, Qi Yang +6 more

cs.RO · · cited 0× · arXiv ↗ · more like this ·

Catching flying objects with a cushioning process is a skill commonly performed by humans, yet it remains a significant challenge for robots. In this paper, we present a framework that combines optimization and learning to achieve compliant catching on mobile manipulators (CCMM). First, we propose a high-level capture planner for mobile manipulators (MM) that calculates the optimal capture point and joint configuration. Next, the pre-catching (PRC) planner ensures the robot reaches the target joint configuration as quickly as possible. To learn compliant catching strategies, we propose a network that leverages the strengths of LSTM for capturing temporal dependencies and positional encoding for spatial context (P-LSTM). This network is designed to effectively learn compliant strategies from human demonstrations. Following this, the post-catching (POC) planner tracks the compliant…

Communication Characteristics of Chinese Fashion Live Commerce - Focused on Dongdaemun Live Commerce Channels -

서울대학교 의류학과 박사과정, Keyu Fan, Jisoo Ha

Cultural Studies · Journal of The Korean Society of Fashion Design · · cited 0× · DOI ↗ · more like this ·

팬데믹의 유행으로 인해 비대면 소비가 전 세계적으로 일상화되고 있으며 라이브 커머스가 전자상거래의 새로운 비즈니스 모델로 주목받고 있다. 본 연구는 한국 동대문에서 진행되는 중국 패션 라이브 커머스의 형식, 진행 방식 그리고 콘텐츠에 대해 고찰하는 데에 목적이 있다. 연구 방법은 네트노그래피에 근거하여 문헌 연구와 참여 관찰을 활용한 사례연구를 병행하였다. 본 연구에서는 중국의 디엔타오 앱을 통해 한국에 서 동대문 패션 제품을 판매하는 라이브 커머스 채널을 대상으로 참여 관찰을 실시하였다. 참여 관찰을 통해 동대문 방송 채널은 방송 장소에 따른 사무실 유형, 오프라인 매장 유형, 사무실과 매장 교대 유형 등 3가지 로 분류되어 각 유형의 특성을 분석하였으며 판매 과정 및 마케팅 전략에 대해 고찰하였다. 그중 6개의 채널 을 심층 분석 대상으로 선정하여 진행자의 커뮤니케이션 특성을 K-fashion에 대한 전문성, 사회적 매력에 의 한 친밀성, 즉시성으로 인한 오락성 등으로 분석하였다. 동대문 라이브 커머스 진행자는 정보를 일방적으로 전달하는 것뿐만 아니라 실시간 질문에 답하며 동시에 시청자들이 다시 질문하도록 유도함으로써 쌍방향 소통을 촉진한다. 진행자들이 패션 제품의 원단, 사이즈, 제작, 색상, 디자인 등 정보를 정확하게 전달하고, 다양한 스타일링을 선보이는 것은 그들의 전문성을 보여준다. 진행자는 여성이 관심 있는 주제로 시청자와 대화를 나누면서 친밀도와 호감도를 증진시키며, 또한 라이브 방송의 즉각적인 특성으로 인해 진행자의 행동 이 드라마틱하고 오락적인 장면으로 연출되는 것이 관찰되었다. 본 연구의 결과는 한국에서 진행되는 중국 라이브 커머스의 현황을 파악하여 국제 무역 활동의 실증적인 연구 자료를 제공하는 데에 학술적 의의를 가질 것이며 한국 패션 제품의 라이브 커머스를…

GGD-SLAM: Monocular 3DGS SLAM Powered by Generalizable Motion Model for Dynamic Environments

Yi Liu, Haoxuan Xu, Hongbo Duan +5 more

cs.RO · · arXiv ↗ · more like this ·

Visual SLAM algorithms achieve significant improvements through the exploration of 3D Gaussian Splatting (3DGS) representations, particularly in generating high-fidelity dense maps. However, they depend on a static environment assumption and experience significant performance degradation in dynamic environments. This paper presents GGD-SLAM, a framework that employs a generalizable motion model to address the challenges of localization and dense mapping in dynamic environments - without predefined semantic annotations or depth input. Specifically, the proposed system employs a First-In-First-Out (FIFO) queue to manage incoming frames, facilitating dynamic semantic feature extraction through a sequential attention mechanism. This is integrated with a dynamic feature enhancer to separate static and dynamic components. Additionally, to minimize dynamic distractors' impact on the static…

Attention Sink in Transformers: A Survey on Utilization, Interpretation, and Mitigation

Zunhai Su, Hengyuan Zhang, Wei Wu +24 more

cs.LG · · arXiv ↗ · more like this ·

As the foundational architecture of modern machine learning, Transformers have driven remarkable progress across diverse AI domains. Despite their transformative impact, a persistent challenge across various Transformers is Attention Sink (AS), in which a disproportionate amount of attention is focused on a small subset of specific yet uninformative tokens. AS complicates interpretability, significantly affecting the training and inference dynamics, and exacerbates issues such as hallucinations. In recent years, substantial research has been dedicated to understanding and harnessing AS. However, a comprehensive survey that systematically consolidates AS-related research and offers guidance for future advancements remains lacking. To address this gap, we present the first survey on AS, structured around three key dimensions that define the current research landscape: Fundamental…

XStreamVGGT: Extremely Memory-Efficient Streaming Vision Geometry Grounded Transformer with KV Cache Compression

Zunhai Su, Weihao Ye, Hansen Feng +5 more

cs.CV · · arXiv ↗ · more like this ·

Learning-based 3D visual geometry models have benefited substantially from large-scale transformers. Among these, StreamVGGT leverages frame-wise causal attention for strong streaming reconstruction, but suffers from unbounded KV cache growth, leading to escalating memory consumption and inference latency as input frames accumulate. We propose XStreamVGGT, a tuning-free approach that systematically compresses the KV cache through joint pruning and quantization, enabling extremely memory-efficient streaming inference. Specifically, redundant KVs originating from multi-view inputs are pruned through efficient token importance identification, enabling a fixed memory budget. Leveraging the unique distribution of KV tensors, we incorporate KV quantization to further reduce memory consumption. Extensive evaluations show that XStreamVGGT achieves mostly negligible performance degradation while…

GEN3D: Generating Domain-Free 3D Scenes from a Single Image

Yuxin Zhang, Ziyu Lu, Hongbo Duan +5 more

cs.CV · · arXiv ↗ · more like this ·

Despite recent advancements in neural 3D reconstruction, the dependence on dense multi-view captures restricts their broader applicability. Additionally, 3D scene generation is vital for advancing embodied AI and world models, which depend on diverse, high-quality scenes for learning and evaluation. In this work, we propose Gen3d, a novel method for generation of high-quality, wide-scope, and generic 3D scenes from a single image. After the initial point cloud is created by lifting the RGBD image, Gen3d maintains and expands its world model. The 3D scene is finalized through optimizing a Gaussian splatting representation. Extensive experiments on diverse datasets demonstrate the strong generalization capability and superior performance of our method in generating a world model and Synthesizing high-fidelity and consistent novel views.

Reinforcement Learning Meets Masked Generative Models: Mask-GRPO for Text-to-Image Generation

Yifu Luo, Xinhao Hu, Keyu Fan +6 more

cs.CV · · arXiv ↗ · more like this ·

Reinforcement learning (RL) has garnered increasing attention in text-to-image (T2I) generation. However, most existing RL approaches are tailored to either diffusion models or autoregressive models, overlooking an important alternative: masked generative models. In this work, we propose Mask-GRPO, the first method to incorporate Group Relative Policy Optimization (GRPO)-based RL into this overlooked paradigm. Our core insight is to redefine the transition probability, which is different from current approaches, and formulate the unmasking process as a multi-step decision-making problem. To further enhance our method, we explore several useful strategies, including removing the KL constraint, applying the reduction strategy, and filtering out low-quality samples. Using Mask-GRPO, we improve a base model, Show-o, with substantial improvements on standard T2I benchmarks and preference…

Principled RL for Flow Matching Emerges from the Chunk-level Policy Optimization

Yifu Luo, Haoyuan Sun, Xinhao Hu +12 more

cs.CV · · arXiv ↗ · more like this ·

Recent Progress in post-training flow matching for text-to-image (T2I) generation with Group Relative Policy Optimization (GRPO) has demonstrated strong potential. However, it is hindered by a critical limitation: inaccurate advantage attribution. In this work, we argue that aggregating consecutive steps into a coherent 'chunk' and shifting the policy optimization paradigm from GRPO's step level to the chunk level can effectively mitigate the negative impact of this issue. Building on this insight, we propose Group Chunking Policy Optimization (GCPO), the first chunk-level reinforcement learning approach for post-training flow matching. Extensive experiments demonstrate that GCPO achieves superior performance on both standard T2I benchmarks and preference alignment, with up to 43% relative gains over GRPO, highlighting the promise of chunk-level policy optimization. The code is…

DyPho-SLAM : Real-time Photorealistic SLAM in Dynamic Environments

Yi Liu, Keyu Fan, Bin Lan +1 more

cs.RO · · arXiv ↗ · more like this ·

Visual SLAM algorithms have been enhanced through the exploration of Gaussian Splatting representations, particularly in generating high-fidelity dense maps. While existing methods perform reliably in static environments, they often encounter camera tracking drift and fuzzy mapping when dealing with the disturbances caused by moving objects. This paper presents DyPho-SLAM, a real-time, resource-efficient visual SLAM system designed to address the challenges of localization and photorealistic mapping in environments with dynamic objects. Specifically, the proposed system integrates prior image information to generate refined masks, effectively minimizing noise from mask misjudgment. Additionally, to enhance constraints for optimization after removing dynamic obstacles, we devise adaptive feature extraction strategies significantly improving the system's resilience. Experiments conducted…

MonoSplat: Generalizable 3D Gaussian Splatting from Monocular Depth Foundation Models

Yifan Liu, Keyu Fan, Weihao Yu +3 more

cs.CV · · arXiv ↗ · more like this ·

Recent advances in generalizable 3D Gaussian Splatting have demonstrated promising results in real-time high-fidelity rendering without per-scene optimization, yet existing approaches still struggle to handle unfamiliar visual content during inference on novel scenes due to limited generalizability. To address this challenge, we introduce MonoSplat, a novel framework that leverages rich visual priors from pre-trained monocular depth foundation models for robust Gaussian reconstruction. Our approach consists of two key components: a Mono-Multi Feature Adapter that transforms monocular features into multi-view representations, coupled with an Integrated Gaussian Prediction module that effectively fuses both feature types for precise Gaussian generation. Through the Adapter's lightweight attention mechanism, features are seamlessly aligned and aggregated across views while preserving…