DragPoser：通过潜空间优化从可变稀疏追踪信号中进行运动重建
DragPoser: Motion Reconstruction from Variable Sparse Tracking Signals via Latent Space Optimization
AI总结：
DragPoser是一种基于深度学习的运动重建系统，通过在结构化潜空间中优化姿态并引入Temporal Predictor网络，实现了高精度、自然且时间连贯的实时运动重建，对可变稀疏追踪信号具有强鲁棒性。
摘要：
遵循用户动作的高质量运动重建可以通过配备许多传感器的高端mocap系统来实现。然而，使用更少的输入设备获得这样的动画质量正变得越来越受欢迎，因为它使mocap更接近大众。主要挑战包括基于学习的方法中末端执行器精度的损失，或者基于IK的解决方案中缺乏自然性和平滑性。此外，此类系统通常针对特定数量的追踪器进行精细调优，并且对缺失数据（例如，在传感器被遮挡或发生故障的场景中）高度敏感。为了应对这些挑战，我们引入了DragPoser，这是一种新颖的基于深度学习的运动重建系统，能够准确表示动态变化的硬约束，实现实时且高精度的末端执行器位置精度。这是通过在结构化潜空间内进行姿态优化过程来实现的。我们的系统仅需在大型人体运动数据集上进行一次性训练，然后约束可以被动态地定义为损失，同时通过在潜空间内计算这些损失的梯度来迭代地优化姿态。为了进一步增强我们的方法，我们引入了一个Temporal Predictor网络，该网络采用Transformer架构直接在潜空间内编码时间性。该网络确保姿态优化被限制在有效姿态的流形上，并利用过去的姿态数据来预测时间上连贯的姿态。结果表明，DragPoser在实现精确的末端执行器定位方面超越了基于IK的方法和最新的数据驱动方法，同时能够生成自然的姿态和时间上连贯的运动。此外，我们的系统展示了对动态约束修改的鲁棒性，并表现出对各种输入配置和变化的卓越适应性。
英文摘要：
High-quality motion reconstruction that follows the user's movements can be achieved by high-end mocap systems with many sensors. However, obtaining such animation quality with fewer input devices is gaining popularity as it brings mocap closer to the general public. The main challenges include the loss of end-effector accuracy in learning-based approaches, or the lack of naturalness and smoothness in IK-based solutions. In addition, such systems are often finely tuned to a specific number of trackers and are highly sensitive to missing data e.g., in scenarios where a sensor is occluded or malfunctions. In response to these challenges, we introduce DragPoser, a novel deep-learning-based motion reconstruction system that accurately represents hard and dynamic on-the-fly constraints, attaining real-time high end-effectors position accuracy. This is achieved through a pose optimization process within a structured latent space. Our system requires only one-time training on a large human motion dataset, and then constraints can be dynamically defined as losses, while the pose is iteratively refined by computing the gradients of these losses within the latent space. To further enhance our approach, we incorporate a Temporal Predictor network, which employs a Transformer architecture to directly encode temporality within the latent space. This network ensures the pose optimization is confined to the manifold of valid poses and also leverages past pose data to predict temporally coherent poses. Results demonstrate that DragPoser surpasses both IK-based and the latest data-driven methods in achieving precise end-effector positioning, while it produces natural poses and temporally coherent motion. In addition, our system showcases robustness against on-the-fly constraint modifications, and exhibits exceptional adaptability to various input configurations and changes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
