Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Institute of Artificial Intelligence, Xiamen University(厦门大学人工智能学院);China Star Optoelectronics Technology(中国星电科技);TCL Corporate Research (HK) Co., Ltd.(TCL香港企业研究有限公司);Department of Electrical and Electronic Engineering, The University of Hong Kong(香港大学电子与电气工程系)
Zunhai Su, Weihao Ye, Hansen Feng, Keyu Fan, Jing Zhang, Dahai Yu, Zhengwu Liu, Ngai Wong
备注:
Submission to the Journal of the Society for Information Display
Learning-based 3D visual geometry models have significantly advanced with the advent of large-scale transformers. Among these, StreamVGGT leverages frame-wise causal attention to deliver robust and efficient streaming 3D reconstruction. However, it suffers from unbounded growth in the Key-Value (KV) cache due to the massive influx of vision tokens from multi-image and long-video inputs, leading to increased memory consumption and inference latency as input frames accumulate. This ultimately limits its scalability for long-horizon applications. To address this gap, we propose XStreamVGGT, a tuning-free approach that seamlessly integrates pruning and quantization to systematically compress the KV cache, enabling extremely memory-efficient streaming inference. Specifically, redundant KVs generated from multi-frame inputs are initially pruned to conform to a fixed KV memory budget using an efficient token-importance identification mechanism that maintains full compatibility with high-performance attention kernels (e.g., FlashAttention). Additionally, leveraging the inherent distribution patterns of KV tensors, we apply dimension-adaptive KV quantization within the pruning pipeline to further minimize memory overhead while preserving numerical accuracy. Extensive evaluations show that XStreamVGGT achieves mostly negligible performance degradation while substantially reducing memory usage by 4.42$\times$ and accelerating inference by 5.48$\times$, enabling practical and scalable streaming 3D applications. The code is available at https://github.com/ywh187/XStreamVGGT/.
Shenzhen International Graduate School Tsinghua University(清华大学深圳国际研究生院);Institute of Artificial Intelligence Xiamen University(厦门大学人工智能研究院);China Star Optoelectronics Technology(中国星电科技);TCL Corporate Research (HK) Co Ltd.(TCL香港企业研发中心);Department of Electrical and Electronic Engineering The University of Hong Kong(香港大学电子与电气工程系)
LiveVLM: Efficient Online Video Understanding via Streaming-Oriented KV Cache and Retrieval
School of Computer Science, Shanghai Jiao Tong University(上海交通大学计算机科学学院);College of Intelligent Robotics and Advanced Manufacturing, Fudan University(复旦大学智能机器人与先进制造学院);Guizhou Provincial Laboratory of Big Data, College of Computer Science and Technology, Guizhou University(贵州大数据省实验室,贵州大学计算机科学与技术学院)
StreamCacheVGGT: Streaming Visual Geometry Transformers with Robust Scoring and Hybrid Cache Compression
Peking University(北京大学);Nanjing University of Science and Technology(南京理工大学);University of Science and Technology of China(中国科学技术大学);Zhejiang University(浙江大学);Huawei(华为);Tongji University(同济大学)
Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models
Institute of Digital Twin, Eastern Institute of Technology(数字孪生研究院,东技术学院);Ocean University of China(中国海洋大学);Shanghai Jiao Tong University(上海交通大学);The Hong Kong Polytechnic University(香港理工大学);Munich Center for Machine Learning, LMU Munich(慕尼黑机器学习中心,慕尼黑大学);Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative(宁波空间智能与数字衍生关键实验室)
LOCI: Spatial Linear Memory for Streaming World Models
Institute of Foundation Models, Mohamed bin Zayed University of Artificial Intelligence(基础模型研究所,穆罕默德·本·扎耶德人工智能大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Pinscreen
Pyramid Forcing: Head-Aware Pyramid KV Cache Policy for High-Quality Long Video Generation
Peking University(北京大学);South China University of Technology(华南理工大学);Xinjiang University(新疆大学);Beihang University(北京航空航天大学);Zhongguancun Academy(中关村学院)
Drive3R: Block-wise Streaming Geometric Foundation Models for Autonomous Driving
AutoLab, School of Artificial Intelligence, Shanghai Jiao Tong University(上海交通大学人工智能学院AutoLab);Voyager Research, Didi Chuxing(滴滴出行Voyager研究院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
Accelerating Streaming Video Large Language Models via Hierarchical Token Compression
EPIC Lab, Shanghai Jiao Tong University(上海交通大学EPIC实验室);Sichuan University(四川大学);Huazhong University of Science and Technology(华中科技大学);Sun Yat-sen University(中山大学);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))