arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

超越标量IoU:基于rollout组的结构化验证用于视频时序定位

Beyond Scalar IoU: Structured Verification from Rollout Groups for Video Temporal Grounding

Youngjae Cho, Won Young Jhoo, Jongsuk Kim

arXiv 2610.07601首次发表:更新:

发表机构

PYLER(PYLER)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

提出SUTURE,利用rollout组联合结构(不一致性与覆盖率)改进视频时序定位的验证器,超越标量IoU,在五个基准上提升性能。

AI 中文摘要

带可验证奖励的强化学习(RLVR)为将预训练模型适配到视频时序定位提供了自然框架,在该框架中生成的时序区间可直接与真实区间进行评分。然而,现有的重叠验证器通常独立地对每个rollout进行评分,未利用rollout组的联合结构。我们提出了SUTURE,该方法将验证条件建立在rollout组上,并在两个互补尺度上利用其结构:rollout之间的不一致性控制目标重新加权的强度,而每个位置上的覆盖率决定奖励质量重新分配的位置。我们证明,由此产生的验证器可精确分解为标准IoU项和由rollout组确定的协方差修正项。局部梯度诊断发现,在分析组中,响应倾向于覆盖相对较少受支持的目标区域。在五个时序定位基准上,SUTURE在每个报告的IoU阈值下均提升了定位性能。其训练后的策略在推理轨迹中也表现出较少的视频起始锚定:对于较晚的事件,首次时序提及更常与标注目标重叠。这些结果共同表明,rollout组的联合结构可以支持更具信息量的时序验证器。

英文摘要

Reinforcement learning with verifiable rewards (RLVR) provides a natural framework for adapting pretrained models to video temporal grounding, where generated temporal intervals can be scored directly against ground truth intervals. Yet existing overlap verifiers typically score each rollout independently, leaving the joint structure of the rollout group unused. We introduce SUTURE, which conditions verification on the rollout group and exploits its structure at two complementary scales: disagreement across rollouts controls how strongly the target is reweighted, while coverage at each position determines where reward mass is redistributed. We show that the resulting verifier admits an exact decomposition into the standard IoU term and a covariance correction determined by the rollout group. A local gradient diagnostic finds a preference for responses covering relatively less supported target regions in the analyzed groups. Across five temporal grounding benchmarks, SUTURE improves grounding performance at every reported IoU threshold. Its trained policy also shows less video-start anchoring in reasoning traces: for later events, the first temporal mention more often overlaps the annotated target. Together, these results show that the joint structure of a rollout group can support a more informative temporal verifier.

CommentsPreprint

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑