arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.01543cs.CV
收藏

高效动态查询动作计数

Efficient Action Counting with Dynamic Queries

Xiaoxuan Ma, Zishi Li, Qiuyan Shang, Wentao Zhu, Hai Ci, Yu Qiao, Yizhou Wang

AI总结:

本文提出了一种基于动态查询表示的高效动作计数方法,通过动态更新和跨查询对比学习,显著提升了长视频和未见过动作的计数性能。

摘要:

时间重复计数旨在量化视频中重复的动作循环。现有大多数方法依赖于相似性相关矩阵来表征动作的重复性,但其可扩展性受到二次计算复杂度的限制。在本工作中,我们提出了一种新的方法,采用动作查询表示来以线性计算复杂度局部化重复的动作循环。基于此表示,我们进一步开发了两个关键组件以解决时间重复计数的核心挑战。首先,为了促进开放集动作计数,我们提出在动作查询上进行动态更新。不同于静态动作查询,这种方法动态地将视频特征嵌入到动作查询中,提供了一种更灵活且可推广的表示。其次,为了区分感兴趣的动作和背景噪声动作,我们引入了跨查询对比学习以正则化不同动作查询对应的视频表示。结果,我们的方法在长视频序列、未见过的动作以及不同速度的动作方面显著优于先前的工作。在具有挑战性的RepCountA基准上,我们在OBO精度上比最先进的方法TransRAC高出26.5%,平均误差降低22.7%,计算负担减少94.1%。代码可在https://github.com/lizishi/DeTRC上获得。

英文摘要:

Temporal repetition counting aims to quantify the repeated action cycles within a video. The majority of existing methods rely on the similarity correlation matrix to characterize the repetitiveness of actions, but their scalability is hindered due to the quadratic computational complexity. In this work, we introduce a novel approach that employs an action query representation to localize repeated action cycles with linear computational complexity. Based on this representation, we further develop two key components to tackle the essential challenges of temporal repetition counting. Firstly, to facilitate open-set action counting, we propose the dynamic update scheme on action queries. Unlike static action queries, this approach dynamically embeds video features into action queries, offering a more flexible and generalizable representation. Secondly, to distinguish between actions of interest and background noise actions, we incorporate inter-query contrastive learning to regularize the video representations corresponding to different action queries. As a result, our method significantly outperforms previous works, particularly in terms of long video sequences, unseen actions, and actions at various speeds. On the challenging RepCountA benchmark, we outperform the state-of-the-art method TransRAC by 26.5% in OBO accuracy, with a 22.7% mean error decrease and 94.1% computational burden reduction. Code is available at https://github.com/lizishi/DeTRC.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑