Platypose:校准的零样本多假设3D人体运动估计
Platypose: Calibrated Zero-Shot Multi-Hypothesis 3D Human Motion Estimation
AI总结:
针对单相机3D姿态估计的模糊性,提出基于预训练扩散模型的零样本多假设运动估计框架Platypose,能生成时间一致的3D姿态序列,在多数据集上实现先进校准与竞争力。
摘要:
由于深度、遮挡或关键点噪声带来的固有模糊性,单相机3D姿态估计是一个定义不明确的问题。多假设姿态估计通过提供与2D测量一致的多个3D姿态来解释这种不确定性。当前的研究主要集中在为单帧静态姿态估计生成多个假设或单假设运动估计上。在本研究中,我们关注多假设运动估计这一新任务。多假设运动估计并非简单地将多假设姿态估计应用于多帧,那会忽略帧间的时间相关性。相反,它要求能够生成时间一致样本的分布,这比多假设姿态估计或单假设运动估计更具挑战性。为此,我们引入了Platypose,这是一个使用在3D人体运动序列上预训练的扩散模型进行零样本3D姿态序列估计的框架。Platypose在运动估计的多假设上优于基线方法。此外,在Human3.6M、MPI-INF-3DHP和3DPW的静态姿态上测试时,Platypose也实现了最先进的校准和具有竞争力的关节误差。最后,由于它是零样本的,我们的方法可以灵活地推广到不同的设置,例如多相机推理。
英文摘要:
Single camera 3D pose estimation is an ill-defined problem due to inherent ambiguities from depth, occlusion or keypoint noise. Multi-hypothesis pose estimation accounts for this uncertainty by providing multiple 3D poses consistent with the 2D measurements. Current research has predominantly concentrated on generating multiple hypotheses for single frame static pose estimation or single hypothesis motion estimation. In this study we focus on the new task of multi-hypothesis motion estimation. Multi-hypothesis motion estimation is not simply multi-hypothesis pose estimation applied to multiple frames, which would ignore temporal correlation across frames. Instead, it requires distributions which are capable of generating temporally consistent samples, which is significantly more challenging than multi-hypothesis pose estimation or single-hypothesis motion estimation. To this end, we introduce Platypose, a framework that uses a diffusion model pretrained on 3D human motion sequences for zero-shot 3D pose sequence estimation. Platypose outperforms baseline methods on multiple hypotheses for motion estimation. Additionally, Platypose also achieves state-of-the-art calibration and competitive joint error when tested on static poses from Human3.6M, MPI-INF-3DHP and 3DPW. Finally, because it is zero-shot, our method generalizes flexibly to different settings such as multi-camera inference.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。