边看边想：多轮视频推理的在线流式段级内存
Think While Watching: Online Streaming Segment-Level Memory for Multi-Turn Video Reasoning in Multimodal Large Language Models
AI总结：
本文提出Think While Watching框架，通过段级内存保持多轮视频推理连续性，提升流式视频理解的准确率和效率。
摘要：
多模态大语言模型（MLLMs）在离线视频理解中表现出色，但大多数模型局限于离线推理或具有较弱的在线推理能力，使在持续到达的视频流上进行多轮交互变得困难。现有流式方法通常采用交替的感知-生成范式，这会阻止同时感知和生成，导致流增长时早期记忆衰减，影响长距离依赖建模。我们提出了Think While Watching，一种以内存为锚点的流式视频推理框架，在多轮交互中保持连续的段级内存。我们构建了一个三阶段、多轮链式思考数据集，并采用阶段匹配的训练策略，同时通过段级流式因果掩码和流式位置编码强制严格因果性。在推理过程中，我们引入了一个高效的流水线，使观看和思考重叠，并自适应地选择最佳的注意力后端。在单轮和多轮流式输入协议下，我们的方法均取得了显著成果。基于Qwen3-VL构建，它在StreamingBench上将单轮准确率提高了2.6%，在OVO-Bench上提高了3.79%。在多轮设置中，它保持性能的同时减少了56%的输出标记。代码可在：https://github.com/wl666hhh/Think_While_Watching/上获得。
英文摘要：
Multimodal large language models (MLLMs) have shown strong performance on offline video understanding, but most are limited to offline inference or have weak online reasoning, making multi-turn interaction over continuously arriving video streams difficult. Existing streaming methods typically use an interleaved perception-generation paradigm, which prevents concurrent perception and generation and leads to early memory decay as streams grow, hurting long-range dependency modeling. We propose Think While Watching, a memory-anchored streaming video reasoning framework that preserves continuous segment-level memory during multi-turn interaction. We build a three-stage, multi-round chain-of-thought dataset and adopt a stage-matched training strategy, while enforcing strict causality through a segment-level streaming causal mask and streaming positional encoding. During inference, we introduce an efficient pipeline that overlaps watching and thinking and adaptively selects the best attention backend. Under both single-round and multi-round streaming input protocols, our method achieves strong results. Built on Qwen3-VL, it improves single-round accuracy by 2.6% on StreamingBench and by 3.79% on OVO-Bench. In the multi-round setting, it maintains performance while reducing output tokens by 56%. Code is available at: https://github.com/wl666hhh/Think_While_Watching/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
