arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.09953cs.CV
收藏

不确定性引导的外观-运动关联网络用于分布外动作检测

Uncertainty-Guided Appearance-Motion Association Network for Out-of-Distribution Action Detection

College of Computing and Data Science(计算与数据科学学院);Nanyang Technological University(南洋理工大学)

Xiang Fang, Arvind Easwaran, Blaise Genest

AI总结:

针对分布外动作检测任务,提出不确定性引导的外观-运动关联网络(UAAN),通过融合外观与运动特征并推理时空物体交互,显著优于现有方法。

摘要:

分布外(OOD)检测旨在检测并拒绝具有语义偏移的测试样本,以防止在分布内(ID)数据集上训练的模型产生不可靠的预测。现有工作仅在图像数据集上提取外观特征,无法处理包含大量运动信息的动态多媒体场景。因此,我们针对一个更现实且更具挑战性的OOD检测任务:OOD动作检测(ODAD)。给定一个未裁剪的视频,ODAD首先对ID动作进行分类并识别OOD动作,然后定位ID和OOD动作。为此,本文提出了一种新颖的不确定性引导的外观-运动关联网络(UAAN),该网络同时探索外观特征和运动上下文,以推理用于ODAD的时空物体间交互。首先,我们设计独立的外观和运动分支,以提取相应的面向外观和面向运动的物体表示。在每个分支中,我们构建一个时空图来推理外观引导和运动驱动的物体间交互。然后,我们设计一个外观-运动注意力模块,融合外观和运动特征以进行最终的动作检测。在两个具有挑战性的数据集上的实验结果表明,UAAN显著优于最先进的方法,证明了其有效性。

英文摘要:

Out-of-distribution (OOD) detection targets to detect and reject test samples with semantic shifts, to prevent models trained on in-distribution (ID) dataset from producing unreliable predictions. Existing works only extract the appearance features on image datasets, and cannot handle dynamic multimedia scenarios with much motion information. Therefore, we target a more realistic and challenging OOD detection task: OOD action detection (ODAD). Given an untrimmed video, ODAD first classifies the ID actions and recognizes the OOD actions, and then localizes ID and OOD actions. To this end, in this paper, we propose a novel Uncertainty-Guided Appearance-Motion Association Network (UAAN), which explores both appearance features and motion contexts to reason spatial-temporal inter-object interaction for ODAD.Firstly, we design separate appearance and motion branches to extract corresponding appearance-oriented and motion-aspect object representations. In each branch, we construct a spatial-temporal graph to reason appearance-guided and motion-driven inter-object interaction. Then, we design an appearance-motion attention module to fuse the appearance and motion features for final action detection. Experimental results on two challenging datasets show that UAAN beats state-of-the-art methods by a significant margin, illustrating its effectiveness.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑