arXiv 2512.008912026-02-11cs.CV

通过分层令牌压缩加速流式视频大型语言模型

Accelerating Streaming Video Large Language Models via Hierarchical Token Compression

EPIC Lab, Shanghai Jiao Tong University（上海交通大学EPIC实验室）；Sichuan University（四川大学）；Huazhong University of Science and Technology（华中科技大学）；Sun Yat-sen University（中山大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））

Yiyu Wang, Xuyang Liu, Xiyan Gui, Xinying Lin, Boxue Yang, Chenfei Liao, Tailai Chen, Linfeng Zhang

备注：: Code is avaliable at \url{https://github.com/lern-to-write/STC}

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

STC通过分层令牌压缩技术，显著降低流式视频大语言模型的处理延迟，同时保持高精度。

摘要：

流式视频大型语言模型（VideoLLMs）在各种视频理解任务中展现了出色的性能，但由于处理连续视频流中的密集视觉令牌的高计算成本，它们在实时部署中面临显著挑战。在流式视频场景中，主要瓶颈在于视觉变换器（ViT）编码阶段，其中对时间上相似的帧的冗余处理导致效率低下。此外，LLM预填充过程中膨胀的令牌序列进一步加剧了延迟和内存开销。为了解决这些挑战，我们提出了Streaming Token Compression（STC），一种即插即用的分层框架，能够无缝整合到现有的流式VideoLLMs中，优化ViT编码和LLM预填充阶段以加速处理。STC引入了两个令牌级加速器：STC-Cacher，通过缓存和重用时间上相似的帧的特征来减少ViT编码的开销，以及STC-Pruner，该工具在令牌进入LLM之前压缩视觉令牌序列，仅保留基于空间和时间相关性的最显著令牌。在四个基线流式VideoLLMs上五个基准测试中的广泛实验表明，STC优于其他压缩方法。值得注意的是，STC在ReKV框架上保留了高达99%的准确性，同时将ViT编码延迟和LLM预填充延迟分别减少了24.5%和45.3%。

英文摘要：

Streaming Video Large Language Models (VideoLLMs) have demonstrated impressive performance across various video understanding tasks, but they face significant challenges in real-time deployment due to the high computational cost of processing dense visual tokens from continuous video streams. In streaming video scenarios, the primary bottleneck lies in the Vision Transformer (ViT) encoding stage, where redundant processing of temporally similar frames leads to inefficiency. Additionally, inflated token sequences during LLM pre-filling further exacerbate latency and memory overhead. To address these challenges, we propose \textbf{S}treaming \textbf{T}oken \textbf{C}ompression (\textbf{STC}), a plug-and-play hierarchical framework that seamlessly integrates into existing streaming VideoLLMs, optimizing both ViT encoding and LLM pre-filling stages to accelerate processing. STC introduces two token-level accelerators: \textbf{STC-Cacher}, which reduces ViT encoding overhead by caching and reusing features from temporally similar frames, and \textbf{STC-Pruner}, which compresses the visual token sequence before it enters the LLM, preserving only the most salient tokens based on both spatial and temporal relevance. Extensive experiments on four baseline streaming VideoLLMs across five benchmarks demonstrate that STC outperforms other compression methods. Notably, STC retains up to \textbf{99\%} of accuracy on the ReKV framework while reducing ViT encoding latency and LLM pre-filling latency by \textbf{24.5\%} and \textbf{45.3\%}.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文