arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.06704cs.CV
收藏

FineGym:一种用于细粒度动作理解的分层视频数据集

FineGym: A Hierarchical Video Dataset for Fine-grained Action Understanding

The Chinese University of Hong Kong(香港中文大学)

Dian Shao, Yue Zhao, Bo Dai, Dahua Lin

摘要:

在公共基准上,当前的动作识别技术已经取得了巨大的成功。然而,当用于现实世界的应用中时,例如体育分析,这需要将活动解析成阶段并区分细微不同的动作的能力,它们的性能仍然远远不能令人满意。为了将动作识别提升到一个新的水平,我们开发了FineGym,这是一个建立在体操视频之上的新数据集。与现有的动作识别数据集相比,FineGym在丰富性、质量和多样性方面都很突出。特别是,它提供了一个三级语义层次的动作和子动作级别的时间注释。例如,“平衡木”事件将被注释为从五个集合导出的基本子动作的序列:“leap-jump-hop”、“beam-turns”、“flight-salto”、“flight-handspring”和“foot”,其中每个集合中的子动作将被进一步注释有精细定义的类标签。这种新的粒度级别为动作识别提出了重大挑战,例如如何从连贯动作中解析时间结构,以及如何区分细微不同的动作类。我们系统地研究了这个数据集上的代表性方法,并获得了一些有趣的发现。我们希望这个数据集可以推动行动理解的研究。

英文摘要:

On public benchmarks, current action recognition techniques have achieved great success. However, when used in real-world applications, e.g. sport analysis, which requires the capability of parsing an activity into phases and differentiating between subtly different actions, their performances remain far from being satisfactory. To take action recognition to a new level, we develop FineGym, a new dataset built on top of gymnastic videos. Compared to existing action recognition datasets, FineGym is distinguished in richness, quality, and diversity. In particular, it provides temporal annotations at both action and sub-action levels with a three-level semantic hierarchy. For example, a "balance beam" event will be annotated as a sequence of elementary sub-actions derived from five sets: "leap-jump-hop", "beam-turns", "flight-salto", "flight-handspring", and "dismount", where the sub-action in each set will be further annotated with finely defined class labels. This new level of granularity presents significant challenges for action recognition, e.g. how to parse the temporal structures from a coherent action, and how to distinguish between subtly different action classes. We systematically investigate representative methods on this dataset and obtain a number of interesting findings. We hope this dataset could advance research towards action understanding.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑