发表机构
University of Southern California (USC); Intel Labs; DEVCOM Army Research Office（南加州大学; 英特尔实验室; 陆军研究办公室）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究在线流视频理解中如何保留信息及组织历史记录的挑战，提出FOLIO聚焦语义记忆系统，通过动态聚焦状态更新记忆，结合短期视觉缓冲区与长期语义记忆，实现轻量级混合检索，提升性能并降低流记忆维护成本。
AI 中文摘要
在在线流视频理解中，视频流持续到达且随时可能有查询。由于流帧无界增长，系统必须持续压缩并保留来自观察到的视频前缀的信息，而未来帧和查询未知。核心挑战是决定保留什么信息以及如何组织维护的历史记录。为应对此挑战，我们引入FOLIO，一个无需训练的聚焦语义记忆系统，它更详细地记录流的重要部分，同时保持周围上下文紧凑。流到达时，FOLIO在动态聚焦状态引导下在段级别更新记忆，结合短期视觉缓冲区和围绕观察到的实体组织并链接到视觉证据缓存的长期语义记忆。查询时，轻量级混合检索将结构化记忆上的直接匹配与语义查询扩展相结合。FOLIO实现了最优性能，在OVO - Bench上使用Qwen3 - VL - 8B达到82.0/69.1感知/向后准确率，在StreamingBench上总体准确率为74.5，同时通过为聚焦实体保留详细记录并紧凑存储周围上下文大幅降低了维护流记忆的成本。
英文摘要
In online streaming video understanding, a video stream continues to arrive and queries may be issued at any time. Because streaming frames grow without bound, the system must continuously compress and retain information from the observed video prefix while future frames and future queries remain unknown. The core challenge is deciding what information to retain and how to organize the maintained history: as this history grows with the stream, memory cost increases and many redundant visual details are retained, whereas later queries often depend on specific entities, actions, and their temporal changes. To address this challenge, we introduce FOLIO, a training-free focused semantic memory system that records important parts of the stream in higher detail while keeping surrounding context compact. As the stream arrives, FOLIO updates memory at the segment level, guided by a dynamic focus state, combining a short-term visual buffer with a long-term semantic memory organized around observed entities and linked to a visual-evidence cache. At query time, lightweight hybrid retrieval combines direct matching over the structured memory with semantic query expansion. FOLIO achieves state-of-the-art performance, reaching 82.0/69.1 Perception/Backward accuracy on OVO-Bench with Qwen3-VL-8B and 74.5 overall accuracy on StreamingBench, while substantially reducing the cost of maintaining streaming memory by reserving detailed records for focused entities and storing surrounding context compactly.
Comments28 pages, 5 figures
