视频流思考：VideoLLMs可以同时观看和思考
Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously
AI总结：
VST提出了一种视频流理解的新范式，通过在流式过程中激活推理，提高实时响应和理解能力，同时在多个基准测试中表现出更高的效率和泛化能力。
摘要：
在线视频大语言模型（VideoLLMs）在支持响应式、实时交互中发挥关键作用。现有方法专注于流式感知，缺乏同步的逻辑推理流。然而，直接应用测试时缩放方法会导致不可接受的响应延迟。为了解决这一权衡，我们提出了视频流思考（VST），一种新的流式视频理解范式。它支持一边观看一边思考的机制，在流式过程中激活对输入视频片段的推理。这种设计通过在视频播放过程中摊销LLM推理延迟，提高了及时理解和连贯认知，同时保持实时响应能力。此外，我们引入了一个全面的训练后流水线，该流水线整合了VST-SFT，通过结构上适应离线VideoLLM到因果流式推理，以及VST-RL，通过在多轮视频交互环境中进行自我探索提供端到端改进。此外，我们设计了一个自动化训练数据合成流水线，利用视频知识图谱生成高质量的流式问答对，使用实体-关系基础的流式推理链来强制多证据推理和持续关注视频流。广泛评估显示，VST-7B在在线基准测试中表现强劲，例如在StreamingBench上达到79.5%的准确率，在OVO-Bench上达到59.3%的准确率。同时，VST在离线长格式或推理基准测试中仍具竞争力。与Video-R1相比，VST响应速度提高了15.7倍，并在VideoHolmes上实现了+5.4%的改进，证明了其在多样化视频理解任务中的更高效率和强大泛化能力。代码、数据和模型将在https://github.com/1ranGuan/VST上发布。
英文摘要：
Online Video Large Language Models (VideoLLMs) play a critical role in supporting responsive, real-time interaction. Existing methods focus on streaming perception, lacking a synchronized logical reasoning stream. However, directly applying test-time scaling methods incurs unacceptable response latency. To address this trade-off, we propose Video Streaming Thinking (VST), a novel paradigm for streaming video understanding. It supports a thinking while watching mechanism, which activates reasoning over incoming video clips during streaming. This design improves timely comprehension and coherent cognition while preserving real-time responsiveness by amortizing LLM reasoning latency over video playback. Furthermore, we introduce a comprehensive post-training pipeline that integrates VST-SFT, which structurally adapts the offline VideoLLM to causal streaming reasoning, and VST-RL, which provides end-to-end improvement through self-exploration in a multi-turn video interaction environment. Additionally, we devise an automated training-data synthesis pipeline that uses video knowledge graphs to generate high-quality streaming QA pairs, with an entity-relation grounded streaming Chain-of-Thought to enforce multi-evidence reasoning and sustained attention to the video stream. Extensive evaluations show that VST-7B performs strongly on online benchmarks, e.g. 79.5% on StreamingBench and 59.3% on OVO-Bench. Meanwhile, VST remains competitive on offline long-form or reasoning benchmarks. Compared with Video-R1, VST responds 15.7 times faster and achieves +5.4% improvement on VideoHolmes, demonstrating higher efficiency and strong generalization across diverse video understanding tasks. Code, data, and models will be released at https://github.com/1ranGuan/VST.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
