回溯记忆：通过动态KV缓存内存扩展令牌实现视频流理解
Going Down Memory Lane: Scaling Tokens for Video Stream Understanding with Dynamic KV-Cache Memory
AI总结：
MemStream通过扩展令牌预算和自适应选择策略，提升视频流理解的准确性和细粒度信息保留能力。
摘要：
视频流理解需要模型能够稳健地编码、存储和检索连续视频流中的信息，以支持准确的视频问答（VQA）。现有最先进的方法依赖于键值缓存来积累帧级信息，但每个帧使用有限数量的令牌，导致细粒度视觉细节的丢失。在本工作中，我们提出扩展令牌预算，以实现更细致的时空理解和推理。首先，我们发现当前方法无法处理密集流：其特征编码导致查询帧相似度分数随时间增加，使检索偏向后期帧。为了解决这个问题，我们引入了自适应选择策略，以减少令牌冗余同时保留局部时空信息。我们进一步提出一种无需训练的检索混合专家，利用外部模型更好地识别相关帧。我们的方法MemStream在CG-Bench上比ReKV高出8.0%，在LVBench上高出8.5%，在VideoMME（长）上比ReKV高出2.4%。
英文摘要：
Streaming video understanding requires models to robustly encode, store, and retrieve information from a continuous video stream to support accurate video question answering (VQA). Existing state-of-the-art approaches rely on key-value caching to accumulate frame-level information over time, but use a limited number of tokens per frame, leading to the loss of fine-grained visual details. In this work, we propose scaling the token budget to enable more granular spatiotemporal understanding and reasoning. First, we find that current methods are ill-equipped to handle dense streams: their feature encoding causes query-frame similarity scores to increase over time, biasing retrieval toward later frames. To address this, we introduce an adaptive selection strategy that reduces token redundancy while preserving local spatiotemporal information. We further propose a training-free retrieval mixture-of-experts that leverages external models to better identify relevant frames. Our method, MemStream, achieves +8.0% on CG-Bench, +8.5% on LVBench, and +2.4% on VideoMME (Long) over ReKV with Qwen2.5-VL-7B.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
