arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1909.12828cs.CV
收藏

基于循环模型拟合的三维人体姿态和形状重建

Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop

University of Pennsylvania(宾夕法尼亚大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所)

Nikos Kolotouros, Georgios Pavlakos, Michael J. Black, Kostas Daniilidis

摘要:

基于模型的人体姿态估计目前通过两种不同的范式来实现。基于优化的方法以迭代的方式将参数化身体模型拟合到2D观察结果,从而导致精确的图像模型对准,但通常缓慢且对初始化敏感。相比之下,基于回归的方法使用深度网络直接从像素估计模型参数,往往会提供合理的结果,但不是像素准确的结果,同时需要大量的监督。在这项工作中,我们的关键见解不是研究哪种方法更好,而是这两种范式可以形成强大的合作。来自网络的合理的直接回归估计可以初始化迭代优化,使拟合更快和更准确。类似地,来自迭代优化的像素精确拟合可以充当网络的强监督。这是我们提出的方法SPIN(SMPL优化循环)的核心。深度网络是一个迭代优化程序,它将身体模型拟合到训练循环中的2D关节,然后使用拟合的估计值来监督网络。我们的方法本质上是自我改进的,因为更好的网络估计可以使优化得到更好的解决方案,而更准确的优化拟合可以为网络提供更好的监督。我们证明了我们的方法在不同环境中的有效性,其中3D地面实况是稀缺的,或不可用,我们始终优于最先进的基于模型的姿态估计方法的显着利润率。包含视频、结果和代码的项目网站可以在https://seas.upenn.edu/~nkolot/projects/spin上找到。

英文摘要:

Model-based human pose estimation is currently approached through two different paradigms. Optimization-based methods fit a parametric body model to 2D observations in an iterative manner, leading to accurate image-model alignments, but are often slow and sensitive to the initialization. In contrast, regression-based methods, that use a deep network to directly estimate the model parameters from pixels, tend to provide reasonable, but not pixel accurate, results while requiring huge amounts of supervision. In this work, instead of investigating which approach is better, our key insight is that the two paradigms can form a strong collaboration. A reasonable, directly regressed estimate from the network can initialize the iterative optimization making the fitting faster and more accurate. Similarly, a pixel accurate fit from iterative optimization can act as strong supervision for the network. This is the core of our proposed approach SPIN (SMPL oPtimization IN the loop). The deep network initializes an iterative optimization routine that fits the body model to 2D joints within the training loop, and the fitted estimate is subsequently used to supervise the network. Our approach is self-improving by nature, since better network estimates can lead the optimization to better solutions, while more accurate optimization fits provide better supervision for the network. We demonstrate the effectiveness of our approach in different settings, where 3D ground truth is scarce, or not available, and we consistently outperform the state-of-the-art model-based pose estimation approaches by significant margins. The project website with videos, results, and code can be found at https://seas.upenn.edu/~nkolot/projects/spin.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑