当思维漂移：面向稳健视频推理的证据锚定
When Thinking Drifts: Evidential Grounding for Robust Video Reasoning
AI总结：
针对视频推理中思维链导致的视觉思维漂移问题，提出视觉证据奖励（VER）强化学习框架，在10个基准上达到顶尖性能，实现基于视觉证据的稳健推理。
摘要：
视频推理，即通过多步逻辑使机器从动态视觉内容中进行推断的任务，对于先进人工智能至关重要。尽管思维链（CoT）机制增强了基于文本的任务中的推理能力，但其在视频理解中的应用仍未得到充分探索。本文提出了一项系统性分析，揭示出CoT在视频推理中常常降低性能，生成冗长但具有误导性的内心独白，导致幻觉化的视觉细节和覆盖正确直觉的现象——我们将其称为“视觉思维漂移”。我们通过贝叶斯视角解释这种漂移，认为CoT轨迹往往偏离实际视觉证据，反而放大内部偏见或语言先验，使模型进行故事编造而非基于证据的推理。为应对这一问题，我们引入了视觉证据奖励（VER），一种新颖的强化学习框架，明确奖励生成可验证地锚定于视觉证据的推理轨迹。在10个多样化的视频理解基准上的全面评估表明，我们的Video-VER持续达到顶尖性能。我们的工作揭示了视频中心推理的独特挑战，并鼓励开发能够稳健地将其推断锚定于视觉证据的人工智能——用于不仅“回答前思考”，而且“思考时观看”的大型多模态模型。
英文摘要：
Video reasoning, the task of enabling machines to infer from dynamic visual content through multi-step logic, is crucial for advanced AI. While the Chain-of-Thought (CoT) mechanism has enhanced reasoning in text-based tasks, its application to video understanding remains underexplored. This paper presents a systematic analysis revealing that CoT often degrades performance in video reasoning, generating verbose but misleading internal monologues, and leading to hallucinated visual details and overridden correct intuitions - a phenomenon we term "visual thinking drift". We explain this drift through a Bayesian lens, positing that CoT traces often diverge from actual visual evidence, instead amplifying internal biases or language priors, causing models to storytell rather than engage in grounded reasoning. To counteract this, we introduce Visual Evidence Reward (VER), a novel reinforcement learning framework that explicitly rewards the generation of reasoning traces that are verifiably grounded in visual evidence. Comprehensive evaluation across 10 diverse video understanding benchmarks demonstrates that our Video-VER consistently achieves top performance. Our work sheds light on the distinct challenges of video-centric reasoning and encourages the development of AI that robustly grounds its inferences in visual evidence - for large multimodal models that not only "think before answering", but also "see while thinking".
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
