arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.00616cs.CVcs.AI

暂停与思考:面向视频基础辅助动作建议的数据集与基准

Pause and Think: A Dataset and Benchmark for Video-Grounded Assistive Action Suggestion

Advanced Micro Devices, Inc.(先进微器件公司)

Shivam Singh, Saptarshi Majumder, Pratik Prabhanjan Brahma, Zicheng Liu, Emad Barsoum

AI总结:

提出 pause-and-think-T 数据集和 pause-and-think-B 基准,通过推理监督训练紧凑模型,在视频场景理解与目标规划任务中达到与大型模型相当的性能。

摘要:

最近的视觉语言模型(VLM)在视频中的基础推理、时间一致性和上下文感知规划方面存在困难。我们引入了 pause-and-think-T,一个以推理为中心的训练数据集,鼓励模型暂停、基于视觉证据进行推理,并生成简洁、可操作的响应。该数据集在生成答案之前促进结构化推理,引导模型走向类人、基于场景的辅助。我们在我们的 pause-and-think-B 基准上微调了一个紧凑的 4B 参数模型,并针对上下文理解和目标规划任务进行了评估。该模型在参数比 Qwen3-VL-235B(58.9%)少 59 倍的情况下达到了 58.0% 的准确率,在场景理解上与 GPT-5.2 匹配,并超越了 GPT-4o。除了我们的基准之外,该模型在 EgoThink 和 TempCompass 上也表现出强大的分布外性能,在可操作性、辅助性、属性识别、情境推理和时间顺序方面取得了显著提升,且无需特定基准训练。我们的结果表明,有针对性的推理监督使紧凑模型能够提供可操作的、基于视觉的指导,同时泛化到训练数据之外,而无需进行大规模模型扩展。

英文摘要:

Recent Vision-Language Models (VLMs) struggle with grounded reasoning, temporal consistency, and context aware planning in videos. We introduce pause-and-think-T, a reasoning-centric training dataset that encourages models to pause, reason over visual evidence, and produce concise, actionable responses. The dataset promotes structured reasoning prior to answer generation, guiding models toward human-like, scene-grounded assistance. We fine-tune a compact 4B-parameter model and evaluate it on our pause-and-think-B benchmark targeting contextual understanding and goal planning tasks. The model achieves 58.0% accuracy at 59x fewer parameters than Qwen3-VL-235B (58.9%), matching GPT-5.2 on scene understanding and surpassing GPT-4o. Beyond our benchmark, it also shows strong out-of-distribution performance on EgoThink and TempCompass, with substantial gains in affordance, assistance, attribution recognition, situated reasoning, and temporal order, without benchmark-specific training. Our results indicate that targeted reasoning supervision enables compact models to deliver actionable, visually grounded guidance while generalizing beyond training data, without requiring large-scale model expansion.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑