arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.08760cs.CVcs.AIcs.LG
收藏

基于网格卷积的三维人体姿态提升

3D Human Pose Lifting with Grid Convolution

Intel Labs China(英特尔中国研究院)

Yangyuxuan Kang, Yuyang Liu, Anbang Yao, Shandong Wang, Enhua Wu

摘要:

用于从2D单视图姿态回归3D人体姿态的现有提升网络通常基于图结构的表示学习用线性层构造。与之形成鲜明对比的是,本文提出了网格卷积(GridConv),模仿图像空间中常规卷积运算的智慧。GridConv基于一种新的语义网格变换(SGT),它利用二进制分配矩阵将不规则图形结构的人体姿势映射到一个规则的编织状网格姿势表示上,通过GridConv操作实现逐层特征学习。我们提供了两种实现SGT的方法,包括手工设计和可学习的设计。令人惊讶的是,这两种设计都取得了令人满意的结果,并且可学习的设计更好,证明了这种新的提升表示学习公式的巨大潜力。为了提高GridConv编码上下文线索的能力,我们在卷积内核上引入了一个注意力模块,使网格卷积操作依赖于输入,空间感知和网格特定。我们表明,我们的全卷积网格提升网络在(1)Human3.6M的常规评估和(2)MPI-INF-3DHP的交叉评估下表现优于最先进的方法,具有明显的优势。代码可在https://github.com/OSVAI/GridConv上获得

英文摘要:

Existing lifting networks for regressing 3D human poses from 2D single-view poses are typically constructed with linear layers based on graph-structured representation learning. In sharp contrast to them, this paper presents Grid Convolution (GridConv), mimicking the wisdom of regular convolution operations in image space. GridConv is based on a novel Semantic Grid Transformation (SGT) which leverages a binary assignment matrix to map the irregular graph-structured human pose onto a regular weave-like grid pose representation joint by joint, enabling layer-wise feature learning with GridConv operations. We provide two ways to implement SGT, including handcrafted and learnable designs. Surprisingly, both designs turn out to achieve promising results and the learnable one is better, demonstrating the great potential of this new lifting representation learning formulation. To improve the ability of GridConv to encode contextual cues, we introduce an attention module over the convolutional kernel, making grid convolution operations input-dependent, spatial-aware and grid-specific. We show that our fully convolutional grid lifting network outperforms state-of-the-art methods with noticeable margins under (1) conventional evaluation on Human3.6M and (2) cross-evaluation on MPI-INF-3DHP. Code is available at https://github.com/OSVAI/GridConv

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑