arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.09248cs.CV
收藏

时间感知推理优化用于视频时间定位

Temporal-Aware Reasoning Optimization for Video Temporal Grounding

University of Science and Technology of China(中国科学技术大学)

Minghang Zheng, Zihao Yin, Yi Yang, Yuxin Peng, Yang Liu

AI总结:

提出TaRO框架,通过构造性推理探索和时间敏感性奖励,增强多模态大模型在视频时间定位中的时间感知推理能力,实现最先进性能。

摘要:

多模态大语言模型(MLLMs)在结合强化学习生成推理路径的视频时间定位任务中取得了显著进展。然而,现有模型常产生浅层推理,对精确时间定位的指导有限。这一限制源于(1)低效的随机探索和(2)仅关注答案正确性而忽略推理质量的奖励函数。为解决这些问题,我们提出TaRO(时间感知推理优化),一个明确增强模型时间思考能力的框架。首先,我们引入构造性推理探索,利用预生成的密集描述构建基于显式视觉线索和时间戳的推理路径,实现高质量时间感知推理的高效探索。其次,为评估推理质量,我们设计了时间敏感性奖励。高质量推理应锚定于特定事件和时间戳。如果思考中的事件边界被破坏,该推理应失效,导致推理路径的logit下降。我们利用这一下降作为推理质量的评判。最后,TaRO遵循渐进式课程,从利用该奖励选择更好的构造推理路径开始,演变为自由探索阶段,模型自主生成有效推理。实验表明,TaRO在VTG基准上达到最先进性能。代码见https://github.com/oceanflowlab/TaRO。

英文摘要:

Multi-modal Large Language Models (MLLMs) have achieved remarkable progress in video temporal grounding with reinforcement learning for generating reasoning paths. However, existing models often produce superficial reasoning, which offers limited guidance for precise temporal localization. This limitation stems from (1) inefficient random exploration and (2) reward functions that focus solely on the answer correctness while ignoring reasoning quality. To address these issues, we propose TaRO (Temporal-Aware Reasoning Optimization), a framework that explicitly enhances the model's ability of thinking with time. First, we introduce a Constructive Reasoning Exploration that leverages pre-generated dense captions to construct reasoning paths grounded in explicit visual cues and timestamps, enabling efficient exploration of high-quality time-aware reasoning. Second, to evaluate reasoning quality, we design a Temporal-Sensitivity Reward. High-quality reasoning should be anchored to specific events and timestamps. If the event boundary under thinking is disrupted, such reasoning should become invalid, leading to a drop in the logit of the reasoning path. We utilize this drop as a critique of reasoning quality. Finally, TaRO follows a progressive curriculum, which starts by utilizing this reward to select better constructed reasoning paths, and evolves to a free exploration phase where the model autonomously generates effective reasoning. Experiments demonstrate that TaRO achieves state-of-the-art performance on VTG benchmarks. Code is available at https://github.com/oceanflowlab/TaRO.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑