<i>XStreamVGGT</i> : Extremely Memory‐Efficient Streaming Vision Geometry Grounded Transformer With KV Cache Compression
Zunhai Su, Weihao Ye, Hansen Feng +5 more
ABSTRACT Learning‐based 3D visual geometry models have significantly advanced with the advent of large‐scale transformers. Among these, StreamVGGT leverages frame‐wise causal attention to deliver robust and efficient streaming 3D reconstruction. However, it suffers from unbounded growth in the Key–Value (KV) cache due to the massive influx of vision tokens from multi‐image and long‐video inputs, leading to increased memory consumption and inference latency as input frames accumulate. To address this gap, we propose XStreamVGGT, a tuning‐free approach that seamlessly integrates pruning and quantization to systematically compress the KV cache, enabling extremely memory‐efficient streaming inference. Specifically, redundant KVs generated from multiframe inputs are first pruned to satisfy a fixed KV memory budget. This process employs an efficient token importance identification mechanism…