MultiCounter：未裁剪视频中的多实例动作无关重复计数
MultiCounter: Multiple Action Agnostic Repetition Counting in Untrimmed Videos
AI总结：
针对未裁剪视频中的多实例重复动作计数，本文提出端到端框架 MultiCounter，通过混合时空交互和任务专用头实现多人检测、跟踪与计数，并在 MultiRep 上显著优于 ByteTrack+RepNet。
摘要：
多实例重复动作计数（MRAC）旨在估计未裁剪视频中多个实例所执行重复动作的次数，这类视频常见于运动和锻炼等以人为中心的领域。本文提出 MultiCounter，这是一个完全端到端的深度学习框架，能够同时检测、跟踪并计数多个人类实例的重复动作。具体而言，MultiCounter 包含两个新颖模块：1）用于在连续帧之间高效建立上下文关联的混合时空交互模块；2）用于准确感知周期边界并提升动作无关人类实例泛化能力的任务专用头。我们在由带标注真实视频生成的名为 MultiRep 的合成数据集上训练 MultiCounter。在 MultiRep 数据集上的实验验证了 MRAC 任务的根本挑战，并展示了所提模型的优越性。与将先进跟踪器和单一重复计数器结合的方案 ByteTrack+RepNet 相比，MultiCounter 将 Period-mAP 大幅提升 41.0%，将 AvgMAE 降低 58.6%，并使 AvgOBO 提高至 1.48 倍。这为 MRAC 领域设立了新的基准。此外，MultiCounter 可在商用 GPU 服务器上实时运行，并且对视频中的人类实例数量不敏感。
英文摘要：
Multi-instance Repetitive Action Counting (MRAC) aims to estimate the number of repetitive actions performed by multiple instances in untrimmed videos, commonly found in human-centric domains like sports and exercise. In this paper, we propose MultiCounter, a fully end-to-end deep learning framework that enables simultaneous detection, tracking, and counting of repetitive actions of multiple human instances. Specifically, MultiCounter incorporates two novel modules: 1) mixed spatiotemporal interaction for efficient context correlation across consecutive frames, and 2) task-specific heads for accurate perception of periodic boundaries and generalization for action-agnostic human instances. We train MultiCounter on a synthetic dataset called MultiRep generated from annotated real-world videos. Experiments on the MultiRep dataset validate the fundamental challenge of MRAC tasks and showcase the superiority of our proposed model. Compared to ByteTrack+RepNet, a solution that combines an advanced tracker with a single repetition counter, MultiCounter substantially improves Period-mAP by 41.0%, reduces AvgMAE by 58.6%, and increases AvgOBO 1.48 times. This sets a new benchmark in the field of MRAC. Moreover, MultiCounter runs in real-time on a commodity GPU server and is insensitive to the number of human instances in a video.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
