面向弱监督时序动作定位的概率视觉语言表示
Probabilistic Vision-Language Representation for Weakly Supervised Temporal Action Localization
AI总结：
针对弱监督时序动作定位中任务差异与细粒度运动捕捉难的问题，提出在概率嵌入空间对齐动作与VLP知识的框架，结合分布内外对比学习增强表示，性能超越现有最优方法。
摘要：
弱监督时序动作定位（WTAL）旨在仅使用视频级标注检测未修剪视频中的动作实例。由于现有许多工作基于动作分类标签优化WTAL模型，会遇到任务差异问题（即“通过分类实现定位”）。为解决该问题，近期研究尝试通过视觉语言预训练（VLP）将动作类别名称作为辅助语义知识加以利用，但现有研究仍存在不足：以往方法主要侧重利用语言模型的文本信息，却忽略了动态人体动作与VLP知识在联合空间中的对齐；此外，以往研究采用的确定性表示难以捕捉细粒度人体运动。针对这些问题，我们提出一种新颖框架，在概率嵌入空间中对齐人体动作知识与VLP知识；同时提出分布内与分布间对比学习方法，基于统计相似性增强概率嵌入空间。大量实验与消融研究表明，我们的方法显著优于所有以往的最优（state-of-the-art）方法。代码可在https://github.com/sejong-rcv/PVLR获取。
英文摘要：
Weakly supervised temporal action localization (WTAL) aims to detect action instances in untrimmed videos using only video-level annotations. Since many existing works optimize WTAL models based on action classification labels, they encounter the task discrepancy problem (i.e., localization-by-classification). To tackle this issue, recent studies have attempted to utilize action category names as auxiliary semantic knowledge through vision-language pre-training (VLP). However, there are still areas where existing research falls short. Previous approaches primarily focused on leveraging textual information from language models but overlooked the alignment of dynamic human action and VLP knowledge in a joint space. Furthermore, the deterministic representation employed in previous studies struggles to capture fine-grained human motions. To address these problems, we propose a novel framework that aligns human action knowledge and VLP knowledge in a probabilistic embedding space. Moreover, we propose intra- and inter-distribution contrastive learning to enhance the probabilistic embedding space based on statistical similarities. Extensive experiments and ablation studies reveal that our method significantly outperforms all previous state-of-the-art methods. Code is available at https://github.com/sejong-rcv/PVLR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
