arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.07797cs.CVcs.AIcs.LG
收藏

基于群体感知的行动质量对比回归评估

Group-aware Contrastive Regression for Action Quality Assessment

Tsinghua University(清华大学);State Key Lab of Intelligent Technologies and Systems(智能技术与系统国家重点实验室);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心)

Xumin Yu, Yongming Rao, Wenliang Zhao, Jiwen Lu, Jie Zhou

摘要:

由于视频之间的细微差异和分数的巨大变化,评估动作质量具有挑战性。大多数现有的方法通过从单个视频回归质量分数来解决这个问题,受到大的视频间分数变化的影响。在本文中,我们证明了视频之间的关系可以为训练和推理过程中更准确的动作质量评估提供重要线索。具体地,我们将动作质量评估的问题重新表述为参考具有共享属性的另一视频(例如,类别和难度),而不是学习未引用的分数。根据这一公式,我们提出了一个新的对比回归(Core)框架,通过成对比较来学习相对分数,突出了视频之间的差异,并指导模型学习评估的关键提示。为了进一步利用两个视频之间的相关信息,我们设计了一个组感知回归树,将传统的分数回归转换为两个更容易的子问题:粗到细分类和小区间回归。为了证明CoRe的有效性,我们在三个主流的AQA数据集上进行了广泛的实验,包括AQA-7,MTL-AQA和JIGSAWS。我们的方法远远优于以前的方法,并在所有三个基准上建立了新的最先进的方法。

英文摘要:

Assessing action quality is challenging due to the subtle differences between videos and large variations in scores. Most existing approaches tackle this problem by regressing a quality score from a single video, suffering a lot from the large inter-video score variations. In this paper, we show that the relations among videos can provide important clues for more accurate action quality assessment during both training and inference. Specifically, we reformulate the problem of action quality assessment as regressing the relative scores with reference to another video that has shared attributes (e.g., category and difficulty), instead of learning unreferenced scores. Following this formulation, we propose a new Contrastive Regression (CoRe) framework to learn the relative scores by pair-wise comparison, which highlights the differences between videos and guides the models to learn the key hints for assessment. In order to further exploit the relative information between two videos, we devise a group-aware regression tree to convert the conventional score regression into two easier sub-problems: coarse-to-fine classification and regression in small intervals. To demonstrate the effectiveness of CoRe, we conduct extensive experiments on three mainstream AQA datasets including AQA-7, MTL-AQA and JIGSAWS. Our approach outperforms previous methods by a large margin and establishes new state-of-the-art on all three benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑