Temporal repetition counting aims to quantify the repeated action cycles within a video. The majority of existing methods rely on the similarity correlation matrix to characterize the repetitiveness of actions, but their scalability is hindered due to the quadratic computational complexity. In this work, we introduce a novel approach that employs an action query representation to localize repeated action cycles with linear computational complexity. Based on this representation, we further develop two key components to tackle the essential challenges of temporal repetition counting. Firstly, to facilitate open-set action counting, we propose the dynamic update scheme on action queries. Unlike static action queries, this approach dynamically embeds video features into action queries, offering a more flexible and generalizable representation. Secondly, to distinguish between actions of interest and background noise actions, we incorporate inter-query contrastive learning to regularize the video representations corresponding to different action queries. As a result, our method significantly outperforms previous works, particularly in terms of long video sequences, unseen actions, and actions at various speeds. On the challenging RepCountA benchmark, we outperform the state-of-the-art method TransRAC by 26.5% in OBO accuracy, with a 22.7% mean error decrease and 94.1% computational burden reduction. Code is available at https://github.com/lizishi/DeTRC.
TransRAC: Encoding Multi-scale Temporal Correlation with Transformers for Repetitive Action Counting
ShanghaiTech University(上海科技大学);National University of Singapore(新加坡国立大学);Shanghai Engineering Research Center of Intelligent Vision and Imaging(上海智能视觉与成像工程技术研究中心);Shanghai Engineering Research Center of Energy Efficient and Custom AI IC(上海能效与定制人工智能集成电路工程技术研究中心)
Repetitive Action Counting with Hybrid Temporal Relation Modeling
School of Computer Science and Information Engineering, Hefei University of Technology (HFUT)(合肥工业大学计算机与信息工程学院);Key Laboratory of Knowledge Engineering with Big Data (HFUT), Ministry of Education(合肥工业大学大数据知识工程重点实验室(教育部));Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院);Anhui Zhonghuitong Technology Co., Ltd.(安徽中汇通科技有限公司);School of Information Science and Technology, University of Science and Technology of China(中国科学技术大学信息科学技术学院)
Counting Beyond Instances: A Benchmark for Group-Individual Object Counting
School of Computer Science and Technology, Huazhong University of Science and Technology(华中科技大学计算机科学与技术学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);University of Liverpool(利物浦大学)
LangForce: Bayesian Decomposition of Vision Language Action Models via Latent Action Queries
Huazhong University of Science and Technology(华中科技大学);Beijing Zhongguancun Academy(北京中关村学院);Zhongguancun Institute of Artificial Intelligence(中关村人工智能研究院);Harbin Institute of Technology(哈尔滨工业大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Zhengzhou University(郑州大学);Beihang University(北航);East China Normal University(东华大学);DeepCybot Co., Ltd.(DeepCybot有限公司)
TReViS: Temporal Repetition Structure Aware Video Synthesis for Self-supervised Repetitive Action Counting
AIGO Unit, Istituto Italiano di Tecnologia(意大利技术研究院AIGO单元);Department of Computer Science, University of Verona(维罗纳大学计算机科学系);University of Genoa(热那亚大学)