arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.07628cs.CV
收藏

P-STMO:用于三维人体姿态估计的预训练时空多对一模型

P-STMO: Pre-Trained Spatial Temporal Many-to-One Model for 3D Human Pose Estimation

Peking University(北京大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);University of Chinese Academy of Sciences(中国科学院大学);Peng Cheng Laboratory(鹏城实验室)

Wenkang Shan, Zhenhua Liu, Xinfeng Zhang, Shanshe Wang, Siwei Ma, Wen Gao

摘要:

本文介绍了一种新的预训练时空多对一(P-STMO)模型的二维到三维人体姿态估计任务。为了降低捕获空间和时间信息的难度,我们将此任务分为两个阶段:预训练(第一阶段)和微调(第二阶段)。在第一阶段,提出了一个自我监督的预训练子任务,称为掩蔽姿态建模。输入序列中的人体关节在空间和时间域中被随机掩蔽。利用一般形式的去噪自动编码器来恢复原始的2D姿态,并且编码器能够以这种方式捕获空间和时间依赖性。在第二阶段,预训练的编码器被加载到STMO模型并进行微调。编码器之后是多对一帧聚合器,以预测当前帧中的3D姿态。特别是,MLP块被用作STMO的空间特征提取器,这产生了比其他方法更好的性能。此外,提出了一种时间下采样策略,以减少数据冗余。在两个基准上的大量实验表明,我们的方法优于最先进的方法,具有更少的参数和更少的计算开销。例如,当使用来自CPN的2D姿势作为输入时,我们的P-STMO模型在Human3.6M数据集上实现了42.1mm MPJPE。同时,它带来了1.5-7.1倍的速度比最先进的方法。代码可在https://github.com/paTRICK-swk/P-STMO上获得。

英文摘要:

This paper introduces a novel Pre-trained Spatial Temporal Many-to-One (P-STMO) model for 2D-to-3D human pose estimation task. To reduce the difficulty of capturing spatial and temporal information, we divide this task into two stages: pre-training (Stage I) and fine-tuning (Stage II). In Stage I, a self-supervised pre-training sub-task, termed masked pose modeling, is proposed. The human joints in the input sequence are randomly masked in both spatial and temporal domains. A general form of denoising auto-encoder is exploited to recover the original 2D poses and the encoder is capable of capturing spatial and temporal dependencies in this way. In Stage II, the pre-trained encoder is loaded to STMO model and fine-tuned. The encoder is followed by a many-to-one frame aggregator to predict the 3D pose in the current frame. Especially, an MLP block is utilized as the spatial feature extractor in STMO, which yields better performance than other methods. In addition, a temporal downsampling strategy is proposed to diminish data redundancy. Extensive experiments on two benchmarks show that our method outperforms state-of-the-art methods with fewer parameters and less computational overhead. For example, our P-STMO model achieves 42.1mm MPJPE on Human3.6M dataset when using 2D poses from CPN as inputs. Meanwhile, it brings a 1.5-7.1 times speedup to state-of-the-art methods. Code is available at https://github.com/paTRICK-swk/P-STMO.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑