EgoExo-Fitness：面向自我中心与外心全身动作理解
EgoExo-Fitness: Towards Egocentric and Exocentric Full-Body Action Understanding
AI总结：
本文提出EgoExo-Fitness数据集，含自我中心和外心视角的健身视频及丰富标注，支持动作分类、定位等任务，促进全身动作理解研究。
摘要：
我们提出了EgoExo-Fitness，一个全新的全身动作理解数据集，其特点是包含从同步的自我中心（第一人称）和固定的外心（第三人称）摄像机录制的健身序列视频。与现有的全身动作理解数据集相比，EgoExo-Fitness不仅包含第一人称视角的视频，还提供了丰富的标注。具体而言，提供了两级时间边界，用于定位单个动作视频以及每个动作的子步骤。更重要的是，EgoExo-Fitness引入了用于可解释动作判断的创新标注——包括技术关键点验证、关于动作执行的自然语言评论以及动作质量分数。综合所有这些，EgoExo-Fitness为研究跨“什么”、“何时”和“多好”维度的自我中心和外心全身动作理解提供了新资源。为了促进自我中心和外心全身动作理解的研究，我们在系列任务（即动作分类、动作定位、跨视角序列验证、跨视角技能判定，以及新提出的基于指导的执行验证任务）上构建了基准，并附有详细分析。代码和数据将在https://github.com/iSEE-Laboratory/EgoExo-Fitness/tree/main提供。
英文摘要：
We present EgoExo-Fitness, a new full-body action understanding dataset, featuring fitness sequence videos recorded from synchronized egocentric and fixed exocentric (third-person) cameras. Compared with existing full-body action understanding datasets, EgoExo-Fitness not only contains videos from first-person perspectives, but also provides rich annotations. Specifically, two-level temporal boundaries are provided to localize single action videos along with sub-steps of each action. More importantly, EgoExo-Fitness introduces innovative annotations for interpretable action judgement--including technical keypoint verification, natural language comments on action execution, and action quality scores. Combining all of these, EgoExo-Fitness provides new resources to study egocentric and exocentric full-body action understanding across dimensions of "what", "when", and "how well". To facilitate research on egocentric and exocentric full-body action understanding, we construct benchmarks on a suite of tasks (i.e., action classification, action localization, cross-view sequence verification, cross-view skill determination, and a newly proposed task of guidance-based execution verification), together with detailed analysis. Code and data will be available at https://github.com/iSEE-Laboratory/EgoExo-Fitness/tree/main.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
