arXiv 2603.028722026-03-06cs.CV

Think-as-You-See: 为大视觉-语言模型设计的流式推理链式思维

Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models

Institute of Digital Twin, Eastern Institute of Technology（数字孪生研究院，东技术学院）；Ocean University of China（中国海洋大学）；Shanghai Jiao Tong University（上海交通大学）；The Hong Kong Polytechnic University（香港理工大学）；Munich Center for Machine Learning, LMU Munich（慕尼黑机器学习中心，慕尼黑大学）；Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative（宁波空间智能与数字衍生关键实验室）

Jialiang Zhang, Junlong Tong, Junyan Lin, Hao Wu, Yirong Sun, Yunpu Ma, Xiaoyu Shen

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

TaYS为大视觉-语言模型设计了流式推理链式思维框架，通过并行化生成、流约束训练和解耦的KV缓存，提升视频理解的效率和响应性。

摘要：

大型视觉语言模型（LVLMs）展现出强大的链式思维（CoT）能力，但大多数现有方法假设在推理前有完整的视频数据，这种批量处理方式与现实世界中的视频流不匹配，因为信息是按顺序到达的。受视频数据流式特性启发，我们研究了两种适用于LVLMs的流式推理范式。第一种是交错范式，交替接收帧和生成部分推理，但受限于严格顺序的缓存更新。为更好地匹配流式输入，我们提出了TaYS，一个统一框架，能够实现真正的并发推理。TaYS集成了并行化CoT生成、流约束训练和流并行推理。它进一步采用时间对齐的推理单元、流式注意力掩码和位置编码，以及一个双KV缓存，将视觉编码与文本推理解耦。我们评估了所有范式在Qwen2.5-VL家族中的代表性视频CoT任务，包括事件动态分析、因果推理和主题理解。实验表明，TaYS在性能上始终优于批量和交错基线，提高了推理性能的同时显著减少了时间到第一个token（TTFT）和整体推理延迟。这些结果展示了数据对齐的流式推理在使LVLMs实现高效和响应的视频理解方面的有效性。我们开源代码见https://github.com/EIT-NLP/StreamingLLM/tree/main/TaYS

英文摘要：

Large Vision Language Models (LVLMs) exhibit strong Chain-of-Thought (CoT) capabilities, yet most existing paradigms assume full-video availability before inference, a batch-style process misaligned with real-world video streams where information arrives sequentially. Motivated by the streaming nature of video data, we investigate two streaming reasoning paradigms for LVLMs. The first, an interleaved paradigm, alternates between receiving frames and producing partial reasoning but remains constrained by strictly ordered cache updates. To better match streaming inputs, we propose \textbf{Think-as-You-See (TaYS)}, a unified framework enabling true concurrent reasoning. TaYS integrates parallelized CoT generation, stream-constrained training, and stream-parallel inference. It further employs temporally aligned reasoning units, streaming attention masks and positional encodings, and a dual KV-cache that decouples visual encoding from textual reasoning. We evaluate all paradigms on the Qwen2.5-VL family across representative video CoT tasks, including event dynamics analysis, causal reasoning, and thematic understanding. Experiments show that TaYS consistently outperforms both batch and interleaved baselines, improving reasoning performance while substantially reducing time-to-first-token (TTFT) and overall reasoning delay. These results demonstrate the effectiveness of data-aligned streaming reasoning in enabling efficient and responsive video understanding for LVLMs. We release our code at https://github.com/EIT-NLP/StreamingLLM/tree/main/TaYS

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。