Motion-o:基于轨迹的视频推理
Motion-o: Trajectory-Grounded Video Reasoning
AI总结:
Motion-o通过引入运动链推理,使视频推理模型能够显式且可验证地表示物体运动轨迹,提升动态和轨迹依赖性推理的可监督性。
摘要:
近年来,视频推理模型越来越多地生成时空证据链,以定位特定时间戳的对象。尽管这些轨迹通过明确证据出现的'哪里'和'何时'提高了可解释性,但它们往往忽略了连接观察的运动,即'如何'。这使得动态和轨迹依赖性的断言在缺乏视频支持时难以监督、验证或惩罚。我们正式将这一缺失组件定义为时空轨迹(STT)推理,并引入Motion-o,一种以运动为中心的视觉语言模型(VLM)扩展,使轨迹显式且可验证。Motion-o通过添加运动链推理(MCoT)来增强证据链,该推理通过离散的<motion/>标签表示物体运动,总结方向、速度和尺度变化。为了监督MCoT,我们把稀疏的时空注释密集化为物体轨迹,并从质心位移和框面积变化中推导出运动描述。然后,我们通过互补的奖励进行训练,包括轨迹一致性和视觉接地,包括基于扰动的信号,该信号惩罚在去除时间证据时运动描述未变化的情况。在多个视频理解基准上,Motion-o在不修改架构的情况下持续改进轨迹忠实推理。这些结果表明,显式的运动接口可以通过将隐含动态转换为可验证的证据来补充现有的VLM流程。代码可在https://github.com/ostadabbas/Motion-o获取。
英文摘要:
Recent video reasoning models increasingly produce spatio-temporal evidence chains that localize objects at specific timestamps. While these traces improve interpretability by grounding \emph{where} and \emph{when} evidence appears, they often leave the motion connecting observations, the \textit{how}, implicit. This makes dynamic and trajectory-dependent claims difficult to supervise, verify, or penalize when unsupported by the video. We formalize this missing component as Spatial-Temporal-Trajectory (STT) reasoning and introduce \textbf{Motion-o}, a motion-centric extension to vision-language models (VLMs) that makes trajectories explicit and verifiable. Motion-o augments evidence chains with Motion Chain of Thought (MCoT), a structured pathway that represents object motion through a discrete \texttt{<motion/>} tag summarizing direction, speed, and scale change. To supervise MCoT, we densify sparse spatio-temporal annotations into object tracks and derive motion descriptors from centroid displacement and box-area change. We then train with complementary rewards for trajectory consistency and visual grounding, including a perturbation-based signal that penalizes motion descriptions that remain unchanged when temporal evidence is removed. Across multiple video understanding benchmarks, Motion-o consistently improves trajectory-faithful reasoning without architectural modifications. These results suggest that an explicit motion interface can complement existing VLM pipelines by converting implicit dynamics into verifiable evidence. Code is available at~\href{https://github.com/ostadabbas/Motion-o}{\faGithub\ \texttt{ostadabbas/Motion-o}}.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。