arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.07795cs.CV
收藏

基于视频的物理人体动作捕捉的最优状态动力学估计

Optimal-state Dynamics Estimation for Physics-based Human Motion Capture from Videos

Department of Electrical Engineering, Linköping University(_linköping大学电气工程系);Delft Center for Systems and Control, Delft University of Technology(代尔夫特理工大学系统与控制中心)

Cuong Le, Viktor Johansson, Manon Kok, Bastian Wandt

AI总结:

针对单目视频人体动作捕捉中的抖动与物理不合理问题,提出结合元PD控制器、物理模拟和循环神经网络卡尔曼滤波的在线最优状态动力学估计方法。

摘要:

近年来,基于单目视频的人体动作捕捉取得了显著进展。然而,现代方法往往会产生时间伪影,例如以抖动动作的形式,并且难以实现平滑且物理合理的运动。以内部力和外部力矩的形式显式整合物理规律,有助于缓解这些伪影。当前最先进的方法利用自动PD控制器来预测力矩和反作用力,以便重新模拟输入运动学,即预定义骨架的关节角度。然而,由于物理模型不完善,这些方法通常需要简化假设并对输入运动学进行大量预处理,才能获得良好性能。为此,受神经卡尔曼滤波方法启发,我们提出一种在在线设置中选择性地将物理模型与运动学观测相结合的新方法。我们开发了一个作为元PD控制器的控制回路,用于预测内部关节力矩和外部反作用力,随后进行基于物理的运动模拟。我们引入循环神经网络来实现卡尔曼滤波器,该滤波器精细地平衡运动学输入与模拟运动,从而得到最优状态动力学预测。我们表明,该滤波步骤对于提供在线监督至关重要,有助于平衡各自输入运动的缺陷,因此不仅对捕捉准确的全局运动轨迹很重要,而且对生成物理合理的人体姿态也很重要。与最先进方法相比,所提方法在基于物理的人体姿态估计任务中表现优异,并证明了预测动力学的物理合理性。代码可在 https://github.com/cuongle1206/OSDCap 获取。

英文摘要:

Human motion capture from monocular videos has made significant progress in recent years. However, modern approaches often produce temporal artifacts, e.g. in form of jittery motion and struggle to achieve smooth and physically plausible motions. Explicitly integrating physics, in form of internal forces and exterior torques, helps alleviating these artifacts. Current state-of-the-art approaches make use of an automatic PD controller to predict torques and reaction forces in order to re-simulate the input kinematics, i.e. the joint angles of a predefined skeleton. However, due to imperfect physical models, these methods often require simplifying assumptions and extensive preprocessing of the input kinematics to achieve good performance. To this end, we propose a novel method to selectively incorporate the physics models with the kinematics observations in an online setting, inspired by a neural Kalman-filtering approach. We develop a control loop as a meta-PD controller to predict internal joint torques and external reaction forces, followed by a physics-based motion simulation. A recurrent neural network is introduced to realize a Kalman filter that attentively balances the kinematics input and simulated motion, resulting in an optimal-state dynamics prediction. We show that this filtering step is crucial to provide an online supervision that helps balancing the shortcoming of the respective input motions, thus being important for not only capturing accurate global motion trajectories but also producing physically plausible human poses. The proposed approach excels in the physics-based human pose estimation task and demonstrates the physical plausibility of the predictive dynamics, compared to state of the art. The code is available on https://github.com/cuongle1206/OSDCap

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑