ProxyCap：通过以人为中心的代理到运动学习实现世界空间中的实时单目全身捕捉
ProxyCap: Real-time Monocular Full-body Capture in World Space via Human-Centric Proxy-to-Motion Learning
AI总结：
本文提出ProxyCap，一种以人为中心的代理到运动学习方法，利用2D骨架和3D旋转数据训练网络，实现实时单目全身捕捉，并在世界空间中保持准确性和脚地接触合理性。
摘要：
基于学习的单目运动捕捉方法最近通过以数据驱动的方式进行回归学习，展现出了令人鼓舞的成果。然而，由于数据收集和网络设计方面的挑战，现有解决方案在实现实时全身捕捉的同时，在世界空间中保持准确性仍然具有挑战性。在这项工作中，我们引入了ProxyCap，一种以人为中心的代理到运动学习方案，用于从2D骨架序列和3D旋转运动的代理数据集中学习世界空间运动。这种代理数据使我们能够构建一个具有准确世界空间监督的基于学习的网络，同时缓解泛化问题。为了在世界空间中获得更准确且物理上合理的预测，我们的网络被设计为从以人为中心的角度学习人体运动，这使其能够理解不同相机轨迹下捕捉到的相同运动。此外，我们的网络中提出了一种接触感知的神经运动下降模块，使其能够感知脚与地面的接触以及运动与代理观测之间的错位。借助所提出的基于学习的解决方案，我们展示了第一个实时单目全身捕捉系统，即使使用手持移动相机，也能在世界空间中实现合理的脚与地面接触。我们的项目页面是https://zhangyux15.github.io/ProxyCapV2。
英文摘要：
Learning-based approaches to monocular motion capture have recently shown promising results by learning to regress in a data-driven manner. However, due to the challenges in data collection and network designs, it remains challenging for existing solutions to achieve real-time full-body capture while being accurate in world space. In this work, we introduce ProxyCap, a human-centric proxy-to-motion learning scheme to learn world-space motions from a proxy dataset of 2D skeleton sequences and 3D rotational motions. Such proxy data enables us to build a learning-based network with accurate world-space supervision while also mitigating the generalization issues. For more accurate and physically plausible predictions in world space, our network is designed to learn human motions from a human-centric perspective, which enables the understanding of the same motion captured with different camera trajectories. Moreover, a contact-aware neural motion descent module is proposed in our network so that it can be aware of foot-ground contact and motion misalignment with the proxy observations. With the proposed learning-based solution, we demonstrate the first real-time monocular full-body capture system with plausible foot-ground contact in world space even using hand-held moving cameras. Our project page is https://zhangyux15.github.io/ProxyCapV2.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
