STPro：用于弱监督时空定位的空间与时间渐进式学习
STPro: Spatial and Temporal Progressive Learning for Weakly Supervised Spatio-Temporal Grounding
AI总结：
本文针对弱监督时空视频定位任务提出STPro渐进式学习框架，通过子动作时间课程学习与拥挤度引导空间课程学习两个模块，有效解决了组合动作理解与复杂场景适应问题，在多个基准数据集上取得最优性能。
摘要：
在本研究中，我们探讨了弱监督时空视频定位（WSTVG），这是一项极具挑战性的任务，仅使用文本查询且无边界框监督，即可在视频中对主体进行时空定位。受视觉-语言基础模型最新进展的启发，我们研究了其在WSTVG中的效用，利用其零样本定位能力。然而，我们发现简单的适配缺乏必要的时空定位能力。为弥合这一差距，我们引入了Tubelet Referral Grounding (TRG)，将文本查询连接到tubelet以实现时空预测。尽管前景广阔，TRG在密集场景下处理组合动作理解时仍面临困难。为解决这些局限性，我们提出了STPro，一种具有两个关键模块的新型渐进式学习框架：(1)子动作时间课程学习(SA-TCL)，逐步构建组合动作理解；(2)拥挤度引导空间课程学习(CG-SCL)，通过空间增加任务难度使模型适应复杂场景。STPro在三个基准数据集上取得了最先进的结果，在VidSTG-Declarative上提升了1.0%，在HCSTVG-v1上提升了3.0%。
英文摘要：
In this work we study Weakly Supervised Spatio-Temporal Video Grounding (WSTVG), a challenging task of localizing subjects spatio-temporally in videos using only textual queries and no bounding box supervision. Inspired by recent advances in vision-language foundation models, we investigate their utility for WSTVG, leveraging their zero-shot grounding capabilities. However, we find that a simple adaptation lacks essential spatio-temporal grounding abilities. To bridge this gap, we introduce Tubelet Referral Grounding (TRG), which connects textual queries to tubelets to enable spatio-temporal predictions. Despite its promise, TRG struggles with compositional action understanding and dense scene scenarios. To address these limitations, we propose STPro, a novel progressive learning framework with two key modules: (1) Sub-Action Temporal Curriculum Learning (SA-TCL), which incrementally builds compositional action understanding, and (2) Congestion-Guided Spatial Curriculum Learning (CG-SCL), which adapts the model to complex scenes by spatially increasing task difficulty. STPro achieves state-of-the-art results on three benchmark datasets, with improvements of 1.0% on VidSTG-Declarative and 3.0% on HCSTVG-v1.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
