视频特定查询-键注意力建模用于弱监督时间动作定位
Video-Specific Query-Key Attention Modeling for Weakly-Supervised Temporal Action Localization
AI总结：
提出VQK-Net，通过视频特定查询-键注意力建模学习每个视频每个动作类别的独特查询，实现弱监督时间动作定位，在三个数据集上达到最先进性能。
摘要：
弱监督时间动作定位旨在仅使用视频级动作标签来识别和定位未修剪视频中的动作实例。当人类观看视频时，我们能够根据不同视频场景调整关于动作的抽象层面知识，并检测某些动作是否正在发生。在本文中，我们模仿人类的行为方式，为在视频中定位和识别多个动作带来了新的视角。我们提出了一种名为VQK-Net的网络，采用视频特定的查询-键注意力建模，为每个输入视频的每个动作类别学习一个独特的查询。学习到的查询不仅包含抽象层面的动作知识特征，还能将这些知识适配到目标视频场景中，并将用于沿时间维度检测相应动作的存在。为了更好地学习这些动作类别查询，我们不仅利用当前输入视频的特征，还通过一种新颖的视频特定动作类别查询学习器结合查询相似性损失来利用不同视频之间的相关性。最后，我们在三个常用数据集（THUMOS14、ActivityNet1.2和ActivityNet1.3）上进行了大量实验，并取得了最先进的性能。
英文摘要：
Weakly-supervised temporal action localization aims to identify and localize the action instances in the untrimmed videos with only video-level action labels. When humans watch videos, we can adapt our abstract-level knowledge about actions in different video scenarios and detect whether some actions are occurring. In this paper, we mimic how humans do and bring a new perspective for locating and identifying multiple actions in a video. We propose a network named VQK-Net with a video-specific query-key attention modeling that learns a unique query for each action category of each input video. The learned queries not only contain the actions' knowledge features at the abstract level but also have the ability to fit this knowledge into the target video scenario, and they will be used to detect the presence of the corresponding action along the temporal dimension. To better learn these action category queries, we exploit not only the features of the current input video but also the correlation between different videos through a novel video-specific action category query learner worked with a query similarity loss. Finally, we conduct extensive experiments on three commonly used datasets (THUMOS14, ActivityNet1.2, and ActivityNet1.3) and achieve state-of-the-art performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
