长期体育视频中的时间组成推理方法
Towards Temporal Compositional Reasoning in Long-Form Sports Videos
AI总结:
本文提出SportsTime基准和CoTR方法,通过时间接地证据组成提升体育视频长期推理能力,优于现有多模态大语言模型。
摘要:
针对多模态理解中体育视频的复杂动态人类活动,本文提出SportsTime基准和CoTR方法,通过时间接地证据组成提升体育视频长期推理能力,优于现有多模态大语言模型。
英文摘要:
Sports videos are a challenging domain for multimodal understanding because they involve complex and dynamic human activities. Despite rapid progress in Multimodal Large Language Models (MLLMs), long-horizon reasoning in sports videos remains difficult, as answering questions requires both locating temporally sparse evidence and integrating it into reasoning. We attribute this limitation to two closely coupled factors: insufficient supervision over temporally dispersed evidence, and the lack of methods that require models to identify, localize, and justify temporal evidence. To address these gaps, we introduce SportsTime, a large-scale benchmark for long-form sports video understanding, comprising 14K+ open-ended QA pairs and 50K+ step-wise temporal evidence annotations. Building on SportsTime, we propose Chain-of-Time Reasoning (CoTR), which treats reasoning as a process of temporally grounded evidence composition. Specifically, during training, CoTR introduces a temporal-reward GRPO to encourage temporally grounded reasoning. During inference, it employs an anchor-observe-infer evidence-seeking loop to iteratively localize, verify, and compose temporal evidence before producing the final answer. Experiments demonstrate the usefulness of SportsTime as a benchmark and the effectiveness of CoTR, which consistently improves temporal compositional reasoning and step-wise grounding quality over strong MLLM baselines.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。